SOTA Video Understanding on VideoMME (test) and PapersWithCode

86.9Overall Score

Gemini 2.5-Pro

Updated 3d ago

Evaluation Results

Method
Gemini 2.5-Pro 2025.05	86.9	-	-	-
Gemini 2.5-Flash 2025.05	81.5	-	-	-
Qwen3-VL-235B-A22B 2026.03	79	-	-	-
Qwen3-VL 32B 2025.05	77.3	-	-	-
SiLVR 2025.05	74.2	-	-	-
Qwen2.5-VL-72B 2026.03	73.3	-	-	-
InternVL3.5-241B-A28B 2026.03	72.9	-	-	-
Qwen3-VL 8B 2025.05	71.8	-	-	-
Qwen2.5-VL 7B 2025.05	71.6	-	-	-
Qwen3-VL-8B-Thinking + FrameRepeat 2026.03	67.9	-	-	-
Qwen3-VL-8B-Thinking + Random-K 2026.03	67.3	-	-	-
Qwen3-VL-8B-Thinking 2026.03	67	-	-	-
Qwen3-VL-8B-Thinking + CLIP-K 2026.03	67	-	-	-
Qwen3-VL-8B-Thinking + Bottom-K 2026.03	66.3	-	-	-
PSA 2025.12	65.4	74.8	67.3	54.2
Full Attention 2025.12	65.1	75.2	66.3	53.7
XAttention 2025.12	65.1	74.8	66.1	54.4
SpargeAttention 2025.12	65	74.9	66.3	53.9
Qwen2.5-VL-7B-Instruct + FrameRepeat 2026.03	63.4	-	-	-
Qwen3-VL-8B-Thinking + FrameRepeat 2026.03	63.2	-	-	-
Qwen3-VL-8B-Thinking + CLIP-K 2026.03	62.6	-	-	-
Qwen2.5-VL-7B-Instruct + CLIP-K 2026.03	62.4	-	-	-
Qwen3-VL-8B-Thinking 2026.03	62.4	-	-	-
Qwen3-VL-8B-Thinking + Random-K 2026.03	62.3	-	-	-
Qwen3-VL-8B-Thinking + Bottom-K 2026.03	62	-	-	-
Qwen2.5-VL-7B-Instruct + Random-K 2026.03	61.5	-	-	-
Video-R1 7B 2025.05	61.4	-	-	-
Qwen2.5-VL-7B-Instruct + Bottom-K 2026.03	61.3	-	-	-
Qwen2.5-VL-7B-Instruct + FrameRepeat 2026.03	61	-	-	-
Qwen2.5-VL-7B-Instruct 2026.03	60.9	-	-	-
Qwen2.5-VL-7B-Instruct + Random-K 2026.03	60.3	-	-	-
Qwen2.5-VL-7B-Instruct + CLIP-K 2026.03	60.2	-	-	-
Qwen2.5-VL-7B-Instruct 2026.03	59.6	-	-	-
Qwen2.5-VL-7B-Instruct + Bottom-K 2026.03	59.6	-	-	-