SAT

Benchmarks

Task Name	Dataset Name	SOTA Result
Image Classification	SAT	Accuracy0.9858	56
Mathematical Reasoning	SAT-Math	SAT Math Accuracy97.66	47
Spatial Mental Modeling	SAT-Real	AVG93.9	41
Mathematical Reasoning	SAT	Accuracy98.2	35
Spatial Reasoning	SAT Real	Accuracy (Pass@1)75.3	25
Spatial Understanding	SAT	Score88	23
Spatial Reasoning	SAT	Overall Acc83.3	21
Image Classification	SAT-6 (test)	Accuracy99.84	21
Reasoning	SAT	Accuracy (SAT)97.6	17
Spatial Aptitude	SAT	Accuracy92	17
Data Contamination Detection	SAT	F1 Score79	16
Image Classification	SAT6	Accuracy96.75	16
Off-policy evaluation for classification error	sat	Bias-0.007	15
Spatial Mental Modeling	SAT (synthesized)	EgoM95.4	15
Analogy recognition	SAT	Accuracy60.78	15
Spatial Action Tasks	SAT	Score92	14
Spatial Awareness	SAT	Accuracy76.7	14
Visual Reasoning	SAT	Accuracy76	13
Visual Question Answering	SAT Real	Accuracy84.1	13
Spatial Reasoning	SAT	Val Metric Score87.7	12
Visual Understanding	SAT	Accuracy73.3	11
Spatial Reasoning	SAT ood (test)	Accuracy79.7	11
Analogy Generation	SAT (test)	Accuracy91	11
Analogy Generation	SAT	Accuracy0.91	11
Multimodal Reasoning	SAT	Accuracy59.49	10

Showing 25 of 61 rows