Vision-Language Reasoning

Benchmarks

Dataset Name	SOTA Method	Metric
VL Reasoning Benchmarks	FRISM	MVista Score74	28	1mo ago
VL Reasoning Benchmarks MathVista, MVerse, MathVision, MMMU, R1-OV, MMStar	FRISM	MathVista Acc79.8	25	1mo ago
SQA3D ScanNet scenes (test)	DINOv3 + SpatialBoost	BLEU-154.9	13	25d ago
ScanQA ScanNet scenes (test)	DINOv3 + SpatialBoost	BLEU-143.3	13	25d ago
CODA-LM 1.0 (test)	Qwen2.5-VL-7B + Ours	Barrier79.8	13	1mo ago
CVBench	Qwen3-VL-8B + GRPO	Accuracy86.16	12	1mo ago
MMStar cleaned	Jigsaw + CARE	Score77.59	10	1mo ago
Winoground	LLaVA-1.5 13B	Simple Acc59.88	9	1mo ago
SugarCrepe (test)	Q4 system redistr (prop)	Simple Accuracy62.75	7	1mo ago
NaturalBench (test)	Q4 system redistr (prop)	Simple Accuracy66.02	7	1mo ago
MME (test)		Simple Accuracy78.98	7	1mo ago
HallusionBench (test)	Q4 system redistr (prop)	Simple Accuracy53.31	7	1mo ago
BEAF (test)	Q4 system redistr (prop)	Simple Accuracy88.4	7	1mo ago
SSRBench	SOLE-R1	General Score (SSRBench)85.6	4	18d ago
nuScenes (reasoning)	Qwen1.5-0.5B	BERT F1 Score67	4	1mo ago
Winoground	CLIP	Text Score30.5	4	1mo ago
Vision-Language Reasoning Suite (MathVerse, MathVista, MathVision, MMMU-Pro, We-Math) (test)	PLM-HoneyBee-3B-GRPO	Average Accuracy46.2	3	1mo ago
CLEVR CoGenT (Split B)	NeSyCoCo	Accuracy78.8	3	1mo ago
CLEVR-CoGenT (Split A)	MDETR	Accuracy99.7	3	1mo ago
CLEVR-SYN hard (val)	NeSyCoCo	Accuracy73.4	2	1mo ago
CLEVR-SYN medium (val)	NeSyCoCo	Accuracy81.2	2	1mo ago
CLEVR-SYN easy (val)	NeSyCoCo	Accuracy92.1	2	1mo ago
SelfEval Benchmark (test)	CLIP	Attribute Binding55.4	2	1mo ago

Showing 23 of 23 rows