The benchmark
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| equation_num. (guess) | The benchmark (held-out slice) | Accuracy59 | 6 | |
| equation_num. (deduce) | The benchmark (held-out slice) | Accuracy95.1 | 6 | |
| bit_manipulation | The benchmark (held-out slice) | Accuracy98.9 | 3 | |
| cipher | The benchmark (held-out slice) | Accuracy100 | 3 | |
| gravity | The benchmark (held-out slice) | Accuracy (Gravity)100 | 3 | |
| unit_conversion | The benchmark (held-out slice) | Accuracy100 | 3 | |
| numeral | The benchmark held-out slice | Accuracy100 | 3 | |
| Overall | The benchmark held-out slice | Accuracy85 | 2 |
Showing 8 of 8 rows