Share your thoughts, 1 month free Claude Pro on usSee more

Binary Question Answering on ACaM Synthetic Videos (test)

99Accuracy (Static)

Human Performance

Updated 19d ago

Evaluation Results

Method	Links
Human Performance 2026.07		99	99.64	98.58	98.59	100	100	99.15	99.15
Our SFT Qwen3-VL-4B 2026.07		88.42	82.6	74.44	81.42	70.23	73.95	78.28	78.62
Mega-SaM 2026.07		86	80.2	67.1	-	-	-	-	-
GPT-5 2026.07		78.65	70.75	64.05	79.66	69.88	82.35	70.22	70.2
Qwen3-VL-4B 2026.07		76.03	72.26	59.76	65.38	59.18	72.73	66.58	65.44
ShotVL-7B 2026.07		75.23	76.82	68.08	72.57	56.25	83.19	72.29	72.73
Qwen3-VL-32B 2026.07		75	81.3	65.34	70.83	72.16	78.57	73.31	72.82
Gemini-3-Pro 2026.07		75	63.4	66.67	61.18	80	86.96	68.26	65.42
CamReasoner-7B 2026.07		74.07	78.64	70.15	76.8	77.19	94.23	75.98	76.24
InternVL3.5-8B 2026.07		72.92	59.68	45.09	64	25.4	69.14	54.79	52.17
Our SFT Qwen3-VL-8B 2026.07		70.89	87.18	78.73	79.63	68.75	79.25	80.56	80.69
Gemini-3.1-Pro 2026.07		70.83	66.97	62.96	59.09	71.91	87.91	67.09	64.76
ViPE 2026.07		66.7	58.9	76.6	-	-	-	-	-
Qwen3-VL-8B 2026.07		57.53	77.48	62.99	72.55	61.39	84.78	69.88	69.65
CameraModel-7B 2026.07		57.43	55.21	56.99	52.73	70.97	47.06	56.53	56.21
Random Guess 2026.07		50	50	50	50	50	50	50	50
LLaVa-OV-7B 2026.07		35.56	49.67	31.3	27.27	59.18	72.16	43.81	39.71
InternVL3.5-14B 2026.07		24.56	58.58	60.31	69.16	70.09	77.55	61.01	57.41
Spatial-MLLM 2026.07		23.88	65.63	55.09	64.97	64.38	70.42	60.8	57.06
VLM-3R-7B 2026.07		21.21	35.23	36.03	26.37	19.35	70.73	35.73	31.5
InternVideo2.5-8B 2026.07		18.18	67.5	53.22	68.22	67.65	83.93	61.68	58.06
G2VLM-2B 2026.07		11.11	8.58	9.55	0	7.41	30	9.76	8.99
Qwen2.5-VL-7B 2026.07		10.91	69.96	66.46	67.23	61.95	82.57	66.58	65.29