MMStar

Benchmarks

Task Name	Dataset Name	SOTA Result
Multimodal Understanding	MMStar	Accuracy82	511
Multimodal Evaluation	MMStar	Accuracy69.5	177
Visual Question Answering	MMStar	Accuracy91.9	151
Multimodal Reasoning	MMStar	Accuracy82	143
Multimodal Reasoning	MMStar	Accuracy79.2	102
General image understanding	MMStar	Accuracy72.13	67
Image Understanding	MMStar	Score65.1	56
General Visual Reasoning	MMStar	Accuracy77.5	55
General VQA	MMStar	Accuracy80.1	51
Visual Reasoning	MMStar	Accuracy69	51
General Visual Question Answering	MMStar	Score77.8	50
General Task	MMStar	Accuracy76.2	36
Visual Question Answering	MMStar (test)	Accuracy68	34
General Reasoning	MMStar	Score69.2	32
Multimodal Capability Evaluation	MMStar	Overall Score68.3	31
Multimodal Understanding	MMStar	Average Score68.01	31
Visual Perception	MMStar	Accuracy73.07	30
Perception	MMStar latest (test)	CP67.2	30
Multimodal Reasoning	MMStar	Accuracy75.2	29
Multi-modal Visual Capability	MMStar	Score63.9	29
Multi-modal Reasoning	MMStar	Accuracy63.78	28
Image Reasoning	MMStar	Accuracy71.85	27
Multimodal Understanding	MMStar	Score68.33	26
Mathematical Reasoning	MMStar Math	Accuracy77.2	26
Multimodal Understanding	MMStar (test)	Accuracy71.6	26

Showing 25 of 94 rows