Share your thoughts, 1 month free Claude Pro on usSee more

Video Multiple Choice Question Answering on ACaM real-world videos 1.0 (test)

99Accuracy (Static)

Human Performance

Updated 19d ago

Evaluation Results

Method	Links
Human Performance 2026.07		99	93.07	90.39	90.83	97.1	96.97	93.44	93.14
VLM-3R-7B 2026.07		94.5	20.69	28.55	35.83	51.43	89.39	40.22	35.02
Qwen3-VL-4B 2026.07		93.5	53.47	34.21	61.67	56.52	80.3	53.01	46.96
LLaVa-OV-7B 2026.07		92.5	28.71	25.79	24.17	62.32	75.76	39.55	34.85
Qwen3-VL-32B 2026.07		92	64.11	47.93	64.17	57.97	86.36	61.95	58.66
ShotVL-7B 2026.07		91	60.4	51.07	32.5	68.12	98.48	60.52	57.09
Mega-SaM 2026.07		89.55	74.94	54.16	-	-	-	-	-
InternVL3.5-8B 2026.07		89	48.51	30.58	58.33	40.58	86.36	48.77	46.15
SFT Qwen3-VL-8B 2026.07		85.5	70.05	64.13	84.17	84.06	96.97	72.75	74.28
Qwen3-VL-8B 2026.07		85	65.84	39.5	61.67	79.71	74.24	58.27	55.25
Gemini-3.1-Pro 2026.07		83.92	63.34	64	62.39	76.81	95.38	68.44	67.73
Gemini-3-Pro 2026.07		83.5	64.36	60.84	65	82.61	96.97	68.24	66.83
InternVideo2.5-8B 2026.07		82.5	31.44	28.43	64.17	50.72	92.42	43.51	41.07
Qwen2.5-VL-7B 2026.07		81	33.91	34.71	58.33	66.67	92.42	46.86	45.62
InternVL3.5-14B 2026.07		81	43.56	39.83	63.33	55.07	89.39	51.37	47.21
CamReasoner-7B 2026.07		76.5	41.58	32.89	59.17	39.13	80.3	45.83	44.99
SFT Qwen3-VL-4B 2026.07		76	71.53	64.13	76.67	78.26	93.94	70.83	72.27
ViPE 2026.07		61.81	49.71	73.17	-	-	-	-	-
GPT-5 2026.07		61.5	66.58	54.55	61.67	68.12	80.3	61.2	61.54
Spatial-MLLM 2026.07		59	16.98	31.72	41.67	24.29	77.27	34.22	29.68
CameraModel-7B 2026.07		55.5	68.81	47.93	45.83	91.3	98.48	58.88	60.56
Random Guess 2026.07		24.5	24.28	24	24.54	24.78	25	24.27	24.37
G2VLM-2B 2026.07		9.5	4.52	4.52	7.44	5.71	7.58	5.66	5.24