AQUA
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Math Reasoning | AQuA | Accuracy93.55 | 194 | |
| Mathematical Reasoning | AQUA | Accuracy85.05 | 167 | |
| Arithmetic Reasoning | AQuA (test) | Accuracy74.63 | 72 | |
| Algebraic Reasoning | AQUA | Accuracy91.89 | 65 | |
| Mathematical Reasoning | AQuA | AQuA Exact Match79.92 | 60 | |
| Mathematical Reasoning | AQuA | Accuracy87.01 | 59 | |
| Arithmetic Reasoning | AQUA | Accuracy77.1 | 57 | |
| Multiple-choice Question Answering | AQuA | Accuracy89.45 | 43 | |
| Hallucination Detection | AQuA | AUROC0.7822 | 31 | |
| Marine species classification | AQUA20 (test) | Macro F188.9 | 28 | |
| Symbolic Reasoning | AQUA | Accuracy80.3 | 26 | |
| Reasoning | AQuA | CACC (%)72 | 25 | |
| Algebraic Word Problems | AQuA | Accuracy90.16 | 24 | |
| Mathematical Reasoning | AQuA | ECE2.02 | 24 | |
| Hybrid Reasoning | AQUA (test) | Accuracy78.5 | 24 | |
| Mathematical Reasoning | AQUA (test) | Accuracy72.44 | 18 | |
| Classification | AQUA | ECE2.96 | 16 | |
| Mathematical Reasoning | AQuA | Accuracy (Without Verifier)74 | 16 | |
| Algebraic Reasoning | AQuA | Performance (%)66.36 | 12 | |
| CoT faithfulness detection | AQuA | Accuracy (CoT Faithfulness)77 | 12 | |
| Complex Reasoning | AQuA | Accuracy28.35 | 12 | |
| Mathematical Reasoning | AQuA | FRS96.8 | 9 | |
| Mathematical Reasoning | AQUA | ECE13.76 | 8 | |
| Mathematical Reasoning | AQUA (val) | Tokens at Best Step (K)336 | 7 | |
| Algebraic Reasoning | AQUA (test) | Accuracy30.94 | 6 |
Showing 25 of 33 rows