WeMath
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Mathematical Reasoning | WeMath | Accuracy84 | 317 | |
| Multimodal Math Reasoning | WeMath | Accuracy98.7 | 228 | |
| Multimodal Reasoning | WeMath | Accuracy78 | 199 | |
| Visual Mathematical Reasoning | WeMath | Accuracy98.7 | 193 | |
| Image Reasoning | WeMath | Accuracy71.3 | 34 | |
| Mathematical multi-modal reasoning | WeMath | Pass@185.11 | 30 | |
| Mathematical Reasoning | WeMath 525 samples | Accuracy78.5 | 24 | |
| Visual Mathematical Reasoning | WeMath strict | Accuracy44.8 | 18 | |
| Multimodal Mathematical Reasoning | WeMath mini (test) | Accuracy79.5 | 18 | |
| Step-wise Verification | WeMath | Macro F163.9 | 18 | |
| Multimodal Mathematical Reasoning | WeMath (test) | Accuracy72.15 | 17 | |
| Vision-Language Modeling | WeMath | Accuracy82.6 | 16 | |
| Visual Reasoning | WeMath strict | Score39 | 12 | |
| Visual Mathematical Reasoning | WeMath Loose | Score79 | 10 | |
| Image-Text Reasoning | WeMath | Accuracy68.29 | 8 | |
| Multimodal Mathematical Reasoning | WeMath | WeMath-S Score36.33 | 8 | |
| Multimodal Scientific Reasoning | WeMath | Accuracy71.77 | 8 | |
| Mathematical Reasoning | WeMath (test) | Accuracy82.9 | 6 | |
| Mathematical reasoning | WeMath-L | Score82.19 | 6 | |
| Mathematical reasoning | WeMath-S | Score68.86 | 6 | |
| Multidisciplinary Reasoning | WeMath | Accuracy61.6 | 6 | |
| Mathematical Reasoning | WeMath loose | Accuracy52.1 | 6 | |
| First Incorrect Step Identification | WeMath | FISI F1 Score24.9 | 6 | |
| Multimodal Mathematical Reasoning | WeMath v1.0 (test) | Accuracy31.8 | 5 | |
| Multimodal Mathematical Reasoning | WeMath 19 | Macro Average Score61.52 | 2 |
Showing 25 of 25 rows