Visual Question Answering on MMMU

81.7Accuracy

Gemini 2.5 Pro

Updated 1d ago

Evaluation Results

Method	Links
Gemini 2.5 Pro 2026.04		81.7
Qwen3-VL-32B-Instruct + SpatialBoost 2026.03		76.4
Qwen3-VL-32B-Instruct 2026.03		76
OpenVLThinkerV2 2026.04		71.6
InternVL3-38B + SpatialBoost 2026.03		70.8
GPT-4o 2026.04		70.7
OneThinker-8B 2026.04		70.6
InternVL3-38B 2026.03		70.1
Qwen3-VL GDPO 2026.04		69.8
Qwen3-VL GRPO 2026.04		69.7
ARES-7B 2026.04		67.9
GDRM 2026.04		60.92
Qwen3-VL-Instruct-8B 2026.04		60.2
Beam Search 2026.04		59.78
VisionZero 2026.04		58.8
MM-Eureka-7B 2026.04		57.3
VL-Rethinker-7B 2026.04		56.7
OpenVLThinker-7B 2026.04		55.8
GDRM 2026.04		55.33
GDRM 2026.04		54.78
GDRM 2026.04		54.78
GDRM 2026.04		54.67
Top-p (nucleus) 2026.04		54.56
GDRM 2026.04		54.44
ε-Sampling 2026.04		54.22
GDRM 2026.04		54.22
Top-k 2026.04		54.11
Greedy 2026.04		54.11
Min-p 2026.04		53.78
Vision-G1 2026.04		53.4
Temp’ Only 2026.04		53.22
η-Sampling 2026.04		51.89
FastV 2026.03		36.61
Full KV Caching 2026.03		36.4
AttentionPack 2026.03		36.38
Full KV Caching 2026.03		35.82
FastV 2026.03		35.8
AttentionPack 2026.03		35.72
AttentionPack 2026.03		35.67
Minicache 2026.03		35.52
ScissorHands 2026.03		35.14
H2O 2026.03		35.05
Full KV Caching 2026.03		34.68
AttentionPack 2026.03		34.59
AttentionPack 2026.03		34.4
FastV 2026.03		34.27
Minicache 2026.03		33.75
H2O 2026.03		33.01
AttentionPack 2026.03		32.79
H2O 2026.03		32.15
ScissorHands 2026.03		31.9
ScissorHands 2026.03		31.66
OpenCLIP + Qwen2.5-3B 2026.05		24.4
C-GSPN + Qwen2.5-3B 2026.05		24.2