MultiArith
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Arithmetic Reasoning | MultiArith | Accuracy100 | 324 | |
| Mathematical Reasoning | MultiArith | Accuracy100 | 143 | |
| Arithmetic Reasoning | MultiArith (test) | Accuracy99.5 | 136 | |
| Math Reasoning | MultiArith | Accuracy98.3 | 65 | |
| Math reasoning | MultiArith (test) | Accuracy99.59 | 54 | |
| Mathematical Reasoning | MultiArith | Original Accuracy99 | 40 | |
| Mathematical Reasoning | MultiArith | Accuracy97.18 | 33 | |
| Mathematical reasoning | MultiArith Out of Distribution | Top-1 Accuracy (Maj@1)100 | 30 | |
| Math Reasoning | MultiArith | Accuracy99.3 | 29 | |
| Group Collusive Attack Detection | MultiArith | Detection Accuracy92 | 27 | |
| Question Answering | MultiArith | Accuracy74.3 | 24 | |
| mathematical reasoning | MultiArith | Accuracy99.15 | 19 | |
| Mathematical Reasoning | MultiArith | Pass@1100 | 18 | |
| Arithmetic Reasoning | MultiArith | Accuracy98.5 | 16 | |
| Mathematical Reasoning | MultiArith | Accuracy100 | 16 | |
| Mathematical Reasoning | MultiArith | Accuracy95 | 15 | |
| Follow-up Questioning Consistency | MultiArith (unseen) | Average Success Count18.33 | 12 | |
| Mathematical Reasoning | MultiArith | Accuracy43.33 | 10 | |
| Mathematical Reasoning | MultiArith | Accuracy (Clean)99.44 | 8 | |
| Mathematical Reasoning | MultiArith (val) | Tokens at Best Step (K)1,640 | 7 | |
| Arithmetic Reasoning | MultiArith | Source Library Accuracy99.64 | 5 | |
| Mathematical Reasoning | MultiArith | Pass@1 Rate82.5 | 5 | |
| Arithmetic Reasoning | MultiArith-100 (pilot) | Accuracy100 | 3 | |
| Arithmetic Reasoning | MultiArith numeric-remapping (attack set) | Accuracy100 | 3 | |
| Arithmetic Reasoning | MultiArith baseline original | Correct Count593 | 3 |
Showing 25 of 29 rows