Share your thoughts, 1 month free Claude Pro on usSee more

Multi-modal Question Answering on MMMU (val)

70.7Accuracy

Proprietary API SOTA (Hurst et al., 2024)

Updated 28d ago

Evaluation Results

Method	Links
Proprietary API SOTA (Hurst et al., 2024) 2025.01		70.7
Gemini-1.5-flash + UnAC 2026.05		60.9
GPT4-V + UnAC 2026.05		60.7
GPT4-V + SKETCHPAD 2026.05		59.7
GPT4-V + CCoT 2026.05		58.7
GPT4-V 2026.05		57.2
GPT4-V + SoM 2026.05		57.2
Open-Source SOTA (Chen et al., 2024d) 2025.01		56.2
Gemini-1.5-flash 2026.05		56.1
InternVL2.0-8B + UnAC 2026.05		54.7
InternVL2.0-8B 2026.05		51.8
LLaVA-OneVision-7B + UnAC 2026.05		51
LLaVA-OneVision-7B 2026.05		48.8
IXC-2.5-Chat 2025.01		44.1
IXC-2.5 2025.01		42.9
LLaVA-v1.6-7B + UnAC 2026.05		37.4
LLaVA-v1.6-7B 2026.05		36.9