SVAMP
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Mathematical Reasoning | SVAMP | Accuracy97 | 403 | |
| Mathematical Reasoning | SVAMP (test) | Accuracy94 | 298 | |
| Performance Estimation | SVAMP | MAE0 | 198 | |
| Math Reasoning | SVAMP | Accuracy96.5 | 89 | |
| Arithmetic Reasoning | SVAMP | Accuracy96.01 | 87 | |
| Arithmetic Reasoning | SVAMP (test) | Accuracy98.16 | 84 | |
| Mathematical Reasoning | SVAMP | Accuracy (%)95.2 | 71 | |
| Arithmetic Reasoning | SVAMP | Accuracy (Overall)93.7 | 54 | |
| Hallucination Detection | SVAMP | Mean AUROC80.6 | 50 | |
| Mathematical Reasoning | SVAMP out-of-domain (test) | Accuracy97 | 50 | |
| Mathematical Word Problem Solving | SVAMP | Accuracy96.94 | 38 | |
| Math Word Problem solving | SVAMP | Value Accuracy94.5 | 38 | |
| Mathematical Reasoning | SVAMP (val) | Accuracy85.1 | 36 | |
| Math Word Problems | SVAMP | Accuracy98.68 | 35 | |
| Mathematical Reasoning | SVAMP | Pass@193.1 | 35 | |
| Mathematical Reasoning | SVAMP | Accuracy72.29 | 33 | |
| Short-form generation | SVAMP | AUROC90 | 32 | |
| Speculative Sampling | SVAMP | Average Acceptance Length5.38 | 28 | |
| Mathematical Reasoning | SVAMP | Pass@1 Accuracy94.2 | 27 | |
| Group Collusive Attack Detection | SVAMP | Detection Accuracy92 | 27 | |
| Mathematical Reasoning | SVAMP 8-shot (test) | Accuracy92 | 25 | |
| Math Reasoning | SVAMP | Accuracy78.67 | 25 | |
| Mathematical Reasoning | SVAMP | Avg Forward Passes5.7 | 24 | |
| Mathematical Reasoning | SVAMP | Accuracy94.31 | 21 | |
| Mathematical reasoning | SVAMP (full set) | Exact Match Accuracy71.1 | 20 |
Showing 25 of 86 rows