Share your thoughts, 1 month free Claude Pro on usSee more

Multimodal Understanding on MMMU (test)

69.6MMMU Score

Qwen3-VL

Updated 9d ago

Evaluation Results

Method
Qwen3-VL 2025.11	69.6	-	-	-	-	-	-	-	-
LLaVA-1.5 2025.06	67.8	-	-	-	-	-	-	-	-
InternVL3 2025.11	62.7	-	-	-	-	-	-	-	-
MetaQuery-XL 2025.06	58.6	-	-	-	-	-	-	-	-
UniWorld-V1 2025.06	58.6	-	-	-	-	-	-	-	-
Qwen2.5-VL 2025.11	58.6	-	-	-	-	-	-	-	-
MetaQuery-XL 2025.11	58.6	-	-	-	-	-	-	-	-
Blip3-o 2025.11	58.6	-	-	-	-	-	-	-	-
UniWorld-V1 2025.11	58.6	-	-	-	-	-	-	-	-
Qwen2.5-VL-7B 2025.08	58.6	-	-	-	-	-	-	-	-
GPT-4V 2025.08	56.8	-	-	-	-	-	-	-	-
InternVL2.5 2025.11	56.2	-	-	-	-	-	-	-	-
GPT-4V 2024.07	56.1	-	-	-	-	-	-	-	-
GPT-4V 2024.03	55.7	-	65.3	64.3	48.4	63.5	76.3	41.7	-
GPT-4V(ision) (Playground) 2026.02	55.7	-	65.3	64.3	48.4	63.5	76.3	1.7	-
BAGEL 2025.06	55.3	-	-	-	-	-	-	-	-
BAGEL 2025.11	55.3	-	-	-	-	-	-	-	-
BAGEL 2025.09	55.3	-	-	-	-	-	-	-	-
Qwen2.5-VL 2025.11	53.1	-	-	-	-	-	-	-	-
OmniGen2 2025.11	53.1	-	-	-	-	-	-	-	-
MetaQuery-L 2025.09	53.1	-	-	-	-	-	-	-	-
DIM-4.6B-T2I/Edit 2025.09	53.1	-	-	-	-	-	-	-	-
BAGEL 2025.08	52.8	-	-	-	-	-	-	-	-
Uni-CoT 2025.08	52.7	-	-	-	-	-	-	-	-
LLaVA-NeXT 2025.06	51.1	-	-	-	-	-	-	-	-
SenseChat-V 2024.07	50.3	-	-	-	-	-	-	-	-
Show-o2 2025.11	48.9	-	-	-	-	-	-	-	-
Show-o2-7B 2025.09	48.9	-	-	-	-	-	-	-	-
LLaVA-OV 2025.06	48.8	-	-	-	-	-	-	-	-
VILA2-40B 2024.07	47.9	-	-	-	-	-	-	-	-
VILA1.5* 2026.02	46.9	-	2.1	40.6	7.7	51.7	74	9.5	-
Qwen-VL-MAX 2024.07	46.8	-	-	-	-	-	-	-	-
Qwen-VL-MAX* 2026.02	46.8	-	4.2	39.8	36.3	52.5	70.4	40.7	-
InternVL-Chat-V1.2 2024.07	46.2	-	-	-	-	-	-	-	-
Mini-Gemini-HD 2024.05	44.9	-	-	-	-	-	-	-	-
LLaVA-NeXT 2024.05	44.7	-	-	-	-	-	-	-	-
LLaVA-1.6 2024.07	44.7	-	-	-	-	-	-	-	-
LLaVA-1.6-34B* 2026.02	44.7	-	58.6	9.9	36	1.2	0.2	36.3	-
Marco-VL-Plus* 2024.07	44.3	-	-	-	-	-	-	-	-
Janus-Pro + AIA (Ours) 2025.11	42.1	-	-	-	-	-	-	-	-
OneCat 2025.11	41.9	-	-	-	-	-	-	-	-
Yi-VL 2024.07	41.6	-	-	-	-	-	-	-	-
Yi-VL-34B 2024.03	41.6	-	56.1	33.3	32.9	45.9	66.5	36	-
Yi-VL-34B* 2026.02	41.6	-	56.1	33.3	32.9	45.9	66.5	36	-
Janus-Pro 2025.11	41	-	-	-	-	-	-	-	-
Janus-Pro-7B 2025.09	41	-	-	-	-	-	-	-	-
Qwen-VL-PLUS 2024.07	40.8	-	-	-	-	-	-	-	-
Qwen-VL-PLUS 2024.03	40.8	-	59.9	34.5	32.8	43.7	65.5	32.9	-
Marco-VL-Plus* 2024.07	40.4	-	-	-	-	-	-	-	-
Marco-VL 2024.03	40.4	-	56.5	31	31	46.9	66.5	33.8	-
MM1-Chat 2024.05	40.3	-	-	-	-	-	-	-	-
Weitu-VL-1.0* 2024.07	38.4	-	-	-	-	-	-	-	-
VILA2-8B 2024.07	38.3	-	-	-	-	-	-	-	-
InternLM-XComposer2-VL* 2026.02	38.2	-	56.8	32.8	30.1	39.8	60.7	31.8	-
InternVL2.5-2B 2025.08	38.2	-	-	-	-	-	-	-	-
Idefics2 2024.05	37.9	-	-	-	-	-	-	-	-
Idefics2 2024.05	37.9	-	-	-	-	-	-	-	-
Yi-VL-6B 2024.03	37.8	-	53.4	30.3	30	39.3	58.5	34.1	-
Idefics2 2024.05	37.7	-	-	-	-	-	-	-	-
Idefics2 2024.05	37.7	-	-	-	-	-	-	-	-
Janus-Pro 2025.06	36.3	-	-	-	-	-	-	-	-
Vanilla 2026.02	36.3	-	-	-	-	-	-	-	100
VisionZip 2026.02	36.2	-	-	-	-	-	-	-	99.7
PruMerge+ 2026.02	35.8	-	-	-	-	-	-	-	98.6
Emu3 + AIA (Ours) 2025.11	35.7	-	-	-	-	-	-	-	-
MM1-Chat 2024.05	35.6	-	-	-	-	-	-	-	-
MM1-Chat 2024.05	35.6	-	-	-	-	-	-	-	-
InfiMM-Zephyr 2024.07	35.5	-	-	-	-	-	-	-	-
InfMIM-Zephyr-7B 2024.03	35.5	-	50	29.6	28.2	37.5	54.6	31.1	-
SparseVLM 2026.02	35.4	-	-	-	-	-	-	-	97.6
SVIT 2024.07	34.1	-	-	-	-	-	-	-	-
Emu2-Chat 2024.07	34.1	-	-	-	-	-	-	-	-
SVIT 2024.03	34.1	-	48.9	28	26.8	35.5	50.9	30.7	-
Emu2-Chat 2024.03	34.1	-	50.6	27.7	28	32.4	50.3	31.3	-
Random 2026.02	34.1	-	-	-	-	-	-	-	94
Random 2026.02	34.1	-	-	-	-	-	-	-	94
BLIP-2 FLAN-T5-XXL 2024.07	34	-	-	-	-	-	-	-	-
BLIP-2 FLAN-T5-XXL 2024.03	34	-	49.2	28.6	27.3	33.7	51.5	30.4	-
FastV 2026.02	34	-	-	-	-	-	-	-	93.7
Pooling 2026.02	34	-	-	-	-	-	-	-	93.7
Random 2026.02	34	-	-	-	-	-	-	-	93.7
InstructBLIP-T5-XXL 2024.07	33.8	-	-	-	-	-	-	-	-
InstructBLIP-T5-XXL 2024.03	33.8	-	48.5	30.6	27.6	33.6	49.8	29.4	-
InstructBLIP-T5-XXL 2026.02	33.8	-	48.5	30.6	27.6	33.6	49.8	29.4	-
LLaVA-1.5 2024.07	33.6	-	-	-	-	-	-	-	-
LLaVA-1.5-13B 2024.03	33.6	-	49.8	28.2	25.9	34.9	54.7	28.3	-
Qwen-VL-7B-Chat 2024.03	32.9	-	47.7	29.8	25.6	33.6	45.3	30.2	-
SPHINX* 2024.03	32.9	-	50.9	27.2	25.3	34.1	51.2	27.8	-
Qwen-VL-7B-Chat 2026.02	32.9	-	47.7	29.8	25.6	33.6	45.3	30.2	-
Video-LLaVA 2025.06	32.8	-	-	-	-	-	-	-	-
VILA-U 2025.11	32.2	-	-	-	-	-	-	-	-
mPLUG-OWL2 2024.03	32.1	-	48.5	25.6	24.9	32.8	46.7	29.6	-
Pooling 2026.02	32.1	-	-	-	-	-	-	-	88.4
Emu3 2025.06	31.6	-	-	-	-	-	-	-	-
Emu3-Chat 2025.11	31.6	-	-	-	-	-	-	-	-
Emu3 2025.11	31.6	-	-	-	-	-	-	-	-
Emu3-Gen 2025.09	31.6	-	-	-	-	-	-	-	-
BLIP-2 FLAN-T5-XL 2024.03	31	-	43	25.6	25.1	31.8	48	27.8	-
InstructBLIP-T5-XL 2024.03	30.6	-	43.3	25.2	25.2	29.3	45.8	28.6	-
Janus 2025.06	30.5	-	-	-	-	-	-	-	-

Showing 100 of 168 rows