Share your thoughts, 1 month free Claude Pro on usSee more

Spatio-Temporal Reasoning on V-STaR

27.5Chain1 (When) m tIoU

DEViL

Updated 1mo ago

Evaluation Results

Method
DEViL 2025.12	27.5	34.3	19.7	40.3	37.2	44.5	41.7	16.7	25	33.2	40.7
Open-o3-Video-32B 2025.10	26.4	64.6	-	-	33.6	-	11.9	-	26	37.9	53.7
Ours (4B) 2026.03	25.2	-	-	-	24.8	-	7	-	23.4	-	-
Open-o3-Video-4B 2025.10	25.1	59.7	-	-	31.2	-	8.1	-	23.8	34.6	47.3
Open-o3-Video-8B 2025.10	24.9	61.1	-	-	33.2	-	8.8	-	23.5	35.5	49
Gemini-2-Flash 2025.12	24.5	53	15.9	0.9	4.6	0.6	2.2	15.2	23.8	26.9	35.6
Gemini-2-Flash 2026.03	24.5	-	-	-	4.6	-	2.2	-	23.8	-	-
Open-o3 Video 2026.03	24.5	-	-	-	25.4	-	6	-	24	-	-
Gemini-2-Flash 2025.10	24.5	53	-	-	4.6	-	2.2	-	23.8	26.9	35.6
Open-o3-Video-7B 2025.10	24.5	61	-	-	25.4	-	6	-	24	33.7	46.6
Qwen3-VL-32B 2025.10	24.5	47.5	-	-	30.9	-	7.5	-	25.7	33.9	45.6
Qwen3-VL-8B 2025.10	23.5	42.9	-	-	27.3	-	7.1	-	24.4	28	34.4
Video-LLaMA3 2025.12	23	41.9	19.8	0.1	0.9	0	0.2	20.4	23.1	21.7	27
Video-LLaMA3 2026.03	23	-	-	-	0.9	-	0.2	-	23.1	-	-
Video-LLaMA3 2025.10	23	41.9	-	-	0.9	-	0.2	-	23.1	21.7	27
Qwen3-VL-4B 2026.03	21.3	-	-	-	22.3	-	5	-	18.5	-	-
LLaVA-ST 2025.12	19.9	55	17.4	20.1	22.2	8.2	9.5	15.7	23.4	30.8	40.6
TRACE 2025.12	19.1	17.6	14.1	-	-	-	-	12	17.1	12	13.3
TRACE 2026.03	19.1	-	-	-	0	-	0	-	17.1	-	-
TRACE 2025.10	19.1	17.6	-	-	0	-	0	-	17.1	12	13.3
Qwen3-VL-4B 2025.10	18.6	59.6	-	-	26.8	-	7.7	-	19.2	31.9	43.7
VTimeLLM 2025.12	17.1	41.5	10.9	0	0.21	0	0	2.1	5.96	17.7	22
GPT-4o 2025.12	16.7	60.8	10.4	2.8	6.5	1.2	3	10	12.8	26.8	38.2
GPT-4o 2026.03	16.7	-	-	-	6.5	-	3	-	12.8	-	-
GPT-4o 2025.10	16.7	60.8	-	-	6.5	-	3	-	12.8	26.8	38.2
Qwen2.5-VL-7B 2026.03	15.4	-	-	-	17	-	2.5	-	13.8	-	-
Qwen2.5-VL-7B (base) 2025.10	15.4	33.5	-	-	17	-	2.5	-	13.8	19.3	22.4
VideoChat2 2025.12	13.7	36.2	13.1	0.1	2.5	0.3	1	12.1	12.5	17	20.3
VideoChat2 2026.03	13.7	-	-	-	2.5	-	1	-	12.5	-	-
VideoChat2 2025.10	13.7	36.2	-	-	2.5	-	1	-	12.5	17	20.3
Oryx-1.5 2025.12	13.5	20.5	4.5	2.2	10.1	1	3.5	5.6	14.8	15.1	13.8
Oryx-1.5-7B 2026.03	13.5	-	-	-	10.1	-	3.5	-	14.8	-	-
Oryx-1.5-7B 2025.10	13.5	20.5	-	-	10.1	-	3.5	-	14.8	15.1	13.8
TimeChat 2025.12	12.01	26.4	8.7	-	-	-	-	8.5	13.6	13.1	14.8
Qwen2.5-VL 2025.12	11.5	54.5	8.9	8.4	13.6	1.4	2	5.4	7.6	24	32.4
LLaVA-Video 2025.12	10.5	49.5	6.3	0.2	1.9	0.3	1.3	5.5	12.2	20.8	27.3
LLaVA-Video 2026.03	10.5	-	-	-	1.9	-	1.3	-	12.2	-	-
LLaVA-Video 2025.10	10.5	49.5	-	-	1.9	-	1.3	-	12.2	20.8	27.3
InternVL-2.5 2025.12	8.7	44.2	4.9	0	0.7	0	0.1	3.8	7.8	17.6	24.9
InternVL-2.5-8B 2026.03	8.7	-	-	-	0.7	-	0.1	-	7.8	-	-
InternVL-2.5-8B 2025.10	8.7	44.2	-	-	0.7	-	0.1	-	7.8	17.6	24.9
Sa2VA 2025.12	0.1	16.4	0	34.18	32.3	40.4	37.5	0	0	17.1	20.3
Sa2VA-8B 2026.03	0.1	-	-	-	32.3	-	37.5	-	0	-	-
Sa2VA-8B 2025.10	0.1	16.4	-	-	32.3	-	37.5	-	0	17.1	20.3