Share your thoughts, 1 month free Claude Pro on usSee more

Multimodal Reasoning on MMMU-Pro

55Std-10 Score

Claude-3.5-Sonnet

Updated 1mo ago

Evaluation Results

Method	Links
Claude-3.5-Sonnet 2024.12		55	48	51.5
GPT-4o-20240513 2024.12		54	49.7	51.9
InternVL2.5-78B 2024.12		51.4	45.9	48.6
Gemini-1.5-Pro 2024.12		49.4	44.4	46.9
Qwen2-VL-72B 2024.12		49.2	43.3	46.2
InternVL2.5-38B 2024.12		48	44	46
InternVL2-Llama3-76B 2024.12		41.9	38	40
InternVL2.5-26B 2024.12		41.6	32.6	37.1
InternVL2.5-8B 2024.12		38.2	30.4	34.3
LLaVA-OneVision-72B 2024.12		38	24	31
InternVL2.5-4B 2024.12		36.4	29	32.7
InternVL2-40B 2024.12		36.3	32.1	34.2
VILA-1.5-40B 2024.12		35.9	14.1	25
Qwen2-VL-7B 2024.12		34.1	27	30.5
InternVL2-26B 2024.12		32.8	27.1	30
InternVL2-8B 2024.12		32.5	25.4	29
MiniCPM-V2.6 2024.12		30.2	24.2	27.2
InternVL-Chat-V1.5 2024.12		29.5	19.9	24.7
InternVL2-4B 2024.12		28.2	21.3	24.7
InternVL2.5-2B 2024.12		27.3	20.1	23.7
Phi-3.5-Vision-4B 2024.12		26.3	13.1	19.7
Qwen2-VL-2B 2024.12		25.3	17.2	21.2
InternVL2.5-1B 2024.12		23.3	15.5	19.4
InternVL2-2B 2024.12		21.6	14.9	18.2
InternVL2-1B 2024.12		16	13.6	14.8