Share your thoughts, 1 month free Claude Pro on usSee more

Massive Multi-discipline Multimodal Understanding on MMMU

71.5Accuracy

SPOT-E

Updated 1mo ago

Evaluation Results

Method
SPOT-E 2026.06	71.5	-	-	-	-	-	-
SPOT-E 2026.06	70.4	-	-	-	-	-	-
Gemini-2.5-Flash 2026.06	70	-	-	-	-	-	-
GPT-4o 2026.06	69.2	-	-	-	-	-	-
DART 2025.12	65.5	-	-	-	-	-	-
SPOT-E 2026.06	65.2	-	-	-	-	-	-
Debate with Consensus 2025.12	63.1	-	-	-	-	-	-
Debate with Judge 2025.12	63.1	-	-	-	-	-	-
InternVL3-8B 2026.06	62.7	-	-	-	-	-	-
Qwen3-VL-8B-Instruct 2026.02	62.4	-	-	-	-	-	-
Qwen3-VL-8B-Thinking 2026.02	62.3	-	-	-	-	-	-
SAP 2026.02	62.3	-	-	-	-	-	-
SPOT-E 2026.06	62	-	-	-	-	-	-
DART 2025.12	61.6	-	-	-	-	-	-
Qwen 3.5 2026.05	61.22	-	-	-	-	-	-
SPOT-E 2026.06	61	-	-	-	-	-	-
Self-Consistency (5-way) 2025.12	60.8	-	-	-	-	-	-
Phoenix-VL 1.5 Medium 2026.05	60.78	-	-	-	-	-	-
Debate with Consensus 2025.12	60.5	-	-	-	-	-	-
Debate with Judge 2025.12	60.3	-	-	-	-	-	-
GPT-4o-mini 2026.06	60	-	-	-	-	-	-
SPOT-E 2026.06	59.5	-	-	-	-	-	-
Claude-3-Opus 2026.02	59.4	-	-	-	-	-	-
CoT 2025.12	58.8	-	-	-	-	-	-
Gemini-1.5 2026.02	58.5	-	-	-	-	-	-
SPOT-E 2026.06	58.5	-	-	-	-	-	-
Debate with Judge 2025.12	58.3	-	-	-	-	-	-
Qwen3-VL-8B 2026.06	58	-	-	-	-	-	-
Debate with Consensus 2025.12	57	-	-	-	-	-	-
GPT-4V 2026.02	56.8	-	-	-	-	-	-
Self-Refinement 2025.12	56.5	-	-	-	-	-	-
InternVL2.5-8B 2026.06	56	-	-	-	-	-	-
Qwen2.5-VL-7B 2026.06	55	-	-	-	-	-	-
Debate with Judge 2025.12	54.8	-	-	-	-	-	-
Qwen2-VL 2024.11	54.1	-	-	-	-	-	-
ViperGPT 2025.12	54	-	-	-	-	-	-
Grok-1.5V 2026.02	53.6	-	-	-	-	-	-
Debate with Consensus 2025.12	53.2	-	-	-	-	-	-
Full Cache 2025.11	53.18	-	-	-	-	-	-
H2O 2025.11	53.18	-	-	-	-	-	-
SnapKV 2025.11	53.18	-	-	-	-	-	-
LOOK-M 2025.11	53.18	-	-	-	-	-	-
MEDA 2025.11	53.18	-	-	-	-	-	-
FlashCache 2025.11	53.18	-	-	-	-	-	-
Claude-3-Sonnet 2026.02	53.1	-	-	-	-	-	-
Self-Consistency (5-way) 2025.12	52.8	-	-	-	-	-	-
Self-Consistency (5-way) 2025.12	52.5	-	-	-	-	-	-
FlashCache 2025.11	52.27	-	-	-	-	-	-
SnapKV 2025.11	52.18	-	-	-	-	-	-
LOOK-M 2025.11	52.18	-	-	-	-	-	-
Chameleon 2025.12	51.6	-	-	-	-	-	-
MEDA 2025.11	51.36	-	-	-	-	-	-
H2O 2025.11	51.18	-	-	-	-	-	-
Qwen2.5-VL 7B, Dense 2026.03	50.78	-	-	-	-	-	-
CoT 2025.12	50.7	-	-	-	-	-	-
CoT 2025.12	50.4	-	-	-	-	-	-
StreamingLLM 2025.11	50	-	-	-	-	-	-
StreamingLLM 2025.11	50	-	-	-	-	-	-
Gemma3-27B 2025.12	49.7	-	-	-	-	-	-
LLaVA-OneVision 7B, Dense 2026.03	49.22	-	-	-	-	-	-
GLM-4.5V 2026.05	48.5	-	-	-	-	-	-
Self-Refinement 2025.12	47.2	-	-	-	-	-	-
Self-Refinement 2025.12	46.7	-	-	-	-	-	-
Phi-3-Vision-4B 2025.03	46.1	-	-	-	-	-	-
Gemma3-4B + AuditDM 2025.12	45.2	-	-	-	-	-	-
LWM 2024.11	44.8	-	-	-	-	-	-
Gemma3-12B 2025.12	44.8	-	-	-	-	-	-
Full Cache 2026.04	43.34	-	-	-	-	-	-
PSFT 2025.08	43.33	-	-	-	-	-	-
Llama 4 Maverick 2026.05	43	-	-	-	-	-	-
SMoE (k=2) 2026.06	42.67	-	-	-	-	-	-
SPOT-E 2026.06	42.5	-	-	-	-	-	-
RetentiveKV 2026.04	42.45	-	-	-	-	-	-
SmoothSMoE annealed (k=2) 2026.06	42.44	-	-	-	-	-	-
H2O 2026.04	42.39	-	-	-	-	-	-
Meda 2026.04	42.36	-	-	-	-	-	-
SnapKV 2026.04	42.33	-	-	-	-	-	-
Full Cache 2026.04	42.33	-	-	-	-	-	-
SAINT 2026.04	42.28	-	-	-	-	-	-
Qwen2-VL-2B 2025.03	42.2	-	-	-	-	-	-
LOOK-M 2026.04	42.15	-	-	-	-	-	-
Full Cache 2026.04	42.11	-	-	-	-	-	-
SmoothSMoE (k=2.5) 2026.06	42.11	-	-	-	-	-	-
SmolVLM2-2.2B 2024.12	42	-	-	-	-	-	-
Base 2025.08	41.89	-	-	-	-	-	-
FlorenceVL-3B 2024.12	41.8	-	-	-	-	-	-
MetaMorph 2025.06	41.8	-	-	-	-	-	-
RetentiveKV 2026.04	41.66	-	-	-	-	-	-
RetentiveKV 2026.04	41.57	-	-	-	-	-	-
SnapKV 2026.04	41.38	-	-	-	-	-	-
Meda 2026.04	41.38	-	-	-	-	-	-
LOOK-M 2026.04	41.32	-	-	-	-	-	-
Self-Consistency (5-way) 2025.12	41.3	-	-	-	-	-	-
SAINT 2026.04	41.22	-	-	-	-	-	-
SnapKV 2026.04	41.16	-	-	-	-	-	-
SAINT 2026.04	41.16	-	-	-	-	-	-
LOOK-M 2026.04	41.12	-	-	-	-	-	-
Meda 2026.04	41.1	-	-	-	-	-	-
Qwen2-VL-2B 2025.06	41.1	-	-	-	-	-	-
LaTtE-Flow 2025.06	41.1	-	-	-	-	-	-

Showing 100 of 327 rows