NaturalBench

Benchmarks

Task Name	Dataset Name	SOTA Result
Multimodal Reasoning	NaturalBench	Accuracy82.5	24
General Multimodal Reasoning	NaturalBench	General Score78.62	21
Robustness to Natural Adversarial Examples	NaturalBench	Accuracy9.89	20
Vision-Language Understanding	Naturalbench	General Score13.2	13
Multimodal Understanding	NaturalBench	NaturalBench Score77.2	12
Image-Text Retrieval	NaturalBench Retrieval	T Score71.9	11
Compositional Reasoning	NaturalBench	Accuracy35.5	10
Multiple-choice Question Answering	NaturalBench	Group Accuracy32.5	8
Vision-Language Reasoning	NaturalBench (test)	Simple Accuracy66.02	7
Robustness Evaluation	NaturalBench	GACC33.5	6
Paired-prompt evaluation	NaturalBench	Simple Accuracy67.81	2
Visual Question Answering	NaturalBench	Simple Accuracy0.6946	2

Showing 12 of 12 rows