MathVerse
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Mathematical Reasoning | MathVerse | Accuracy84.1 | 266 | |
| Mathematical Multimodal Reasoning | MathVerse | Accuracy81.4 | 259 | |
| Visual Mathematical Reasoning | MathVerse | Accuracy78.8 | 194 | |
| Multimodal Reasoning | MathVerse | Accuracy68.5 | 138 | |
| Mathematical Reasoning | MathVerse mini | Accuracy82.6 | 89 | |
| Multimodal Mathematical Reasoning | MathVerse | Average Score62.4 | 70 | |
| Mathematical Reasoning | MathVerse Vision Only | Accuracy67 | 58 | |
| Mathematical Reasoning | MathVerse V | Accuracy67.8 | 49 | |
| Multimodal Mathematical Reasoning | MathVerse mini | Accuracy65.9 | 45 | |
| Visual Reasoning | MathVerse | Accuracy61.29 | 40 | |
| Mathematical Visual Question Answering | MathVerse | Accuracy82.9 | 37 | |
| Multimodal Mathematical Reasoning | MathVerse-V | Accuracy81.2 | 33 | |
| Multimodal Mathematical Reasoning | MathVerse (test) | Accuracy (ALL)64.9 | 33 | |
| Visual Mathematical Reasoning | MathVerse VO | BoN@8 Accuracy70.46 | 29 | |
| STEM & Reasoning | MathVerse | Accuracy91.62 | 28 | |
| Mathematical problem solving | MathVerse (testmini) | Accuracy64.9 | 28 | |
| Multimodal Reasoning | MathVerse | Mean@8 Accuracy60.95 | 26 | |
| Multimodal Mathematical Reasoning | MathVerse mini (test) | T-Dominant Score70.94 | 26 | |
| Mathematical reasoning | MathVerse mini (test) | Accuracy66.9 | 26 | |
| Multimodal Reasoning | MathVerse MINI | Accuracy77.7 | 25 | |
| Mathematical Reasoning | MathVerse vision-only (testmini) | Accuracy43 | 22 | |
| Multimodal Reasoning | MathVerse VO | FEI Score48.71 | 20 | |
| Multimodal Autoformalization | MATHVERSE Solid Geometry | Compilation Success80 | 20 | |
| Multimodal Reasoning | MathVerse Vision Only | Accuracy58.76 | 19 | |
| Visual Mathematical Reasoning | MathVerse vision-only | Accuracy54.6 | 18 |
Showing 25 of 84 rows