Image Understanding

Benchmarks

Dataset Name	SOTA Method	Metric
MMStar	GPT-4o-2024-11-20	Score65.1	54	2d ago
TextVQA	MiniCPM-LLaMA3-V 2.5	Accuracy725	40	11d ago
MMBench CN	Vanilla	Accuracy60.6	39	1mo ago
MME		Score2,312	39	1mo ago
Image Understanding Suite (TextVQA, ChartQA, MMStar, MMBench, MMVet, MME, RealWorldQA, COCO)	InternVL-3.5-30B-A3B-HF	TextVQA Score85.76	34	1mo ago
SEED-Bench image	Lumina-DiMOO	Accuracy83.1	27	1mo ago
MMBench		Score83.81	23	11d ago
Image benchmarks Aggregate	MRoPE-I	Overall Score64.82	21	11d ago
COCO		Score69.3	16	1mo ago
MMSI-Bench 68 (test)	Gemini-2.0-Flash	Average Score69.4	12	1mo ago
3DSRBench real 45 (test)	PolyV	Average Score63.4	12	1mo ago
MMStar 11 (test)	PolyV	Average Score71.4	11	1mo ago
MME-P	BAGEL-7B	MME-P Score1,687	11	1mo ago
Image Understanding benchmarks GQA, MME, POPE, VQA^T, MMB, SQA	Vanilla	GQA Score62.2	10	1mo ago
MMBench v1.1 (test)	BAGEL	MMB^i Score85	10	1mo ago
MMEB Image v2	MetaEmbed	Accuracy (CLS)68.1	9	1mo ago
LLaVABenchWilder	LLaVA-OneVision-7B w/ VL-MDR	Score77.9	8	10d ago
LLaVABench		Score101.9	8	10d ago
MMIU	VideoChat-TPO	MMIU Score40.2	7	1mo ago
SEED-Bench 2	VideoChat-TPO	SEED-2 Image Score67.3	6	1mo ago
Jetson Orin Nano Performance Benchmark	Mobile-O-0.5B	Vision Encoding Time (ms)88	4	1mo ago
MacBook M2 Pro	Mobile-O-0.5B	Vision Encoding Time (ms)56	4	1mo ago
COCO GPT-based evaluation	Chat-UniVi	Conversation Score84.1	4	1mo ago
MathVerse	Sparse-LaViDa	Accuracy37.9	2	1mo ago
MathVista	LaViDa-O	Accuracy56.9	2	1mo ago

Showing 25 of 28 rows