Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

MathVerse

Benchmarks

Task NameDataset NameSOTA ResultTrend
Visual Mathematical ReasoningMathVerse
Accuracy69.2
73
Mathematical ReasoningMathVerse mini
Accuracy82.6
50
Mathematical ReasoningMathVerse
Accuracy81.45
39
Multimodal Mathematical ReasoningMathVerse (test)
Accuracy (ALL)64.9
33
Mathematical Multimodal ReasoningMathVerse
Accuracy57.6
29
Mathematical problem solvingMathVerse (testmini)
Accuracy64.9
28
Multimodal Mathematical ReasoningMathVerse mini (test)
T-Dominant Score70.94
26
Multimodal ReasoningMathVerse MINI
Accuracy77.7
25
Multimodal ReasoningMathVerse
Accuracy55.8
20
Multimodal AutoformalizationMATHVERSE Solid Geometry
Compilation Success80
20
Step-wise VerificationMathVerse VO
Macro F162.8
18
Multimodal ReasoningMathVerse (testmini)
Mean@1 Accuracy57.6
17
Multimodal Mathematical ReasoningMathVerse-V
Accuracy81.2
17
Visual Mathematical ReasoningMathVerse mini Vision Only
Avg@3 Score48.6
14
Mathematical ReasoningMathVerse Vision Only
Accuracy53.4
14
Mathematical ReasoningMathVerse-Plus Vision Only 1.0 (test)
Accuracy31
12
Mathematical ReasoningMathVerse-Plus Vision Dominant 1.0 (test)
Accuracy41
12
Mathematical ReasoningMathVerse-Plus Text Dominant 1.0 (test)
Accuracy53
12
Mathematical ReasoningMathVerse-Plus All 1.0 (test)
Accuracy43.6
12
Multimodal Mathematical ReasoningMathVerse mini-vision
Score31.2
12
Multimodal mathematical reasoningMathVerse (vision)
Pass@1 Accuracy76.39
11
General Perception and ReasoningMathVerse VO
Score52
11
Multimodal AutoformalizationMATHVERSE Function
Compile Rate100
10
Multimodal AutoformalizationMATHVERSE Plane Geometry
Compile Rate76
10
Mathematical Visual Question AnsweringMathVerse
Accuracy26.4
8
Showing 25 of 40 rows