BIG-bench Hard
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| General Reasoning | Big-Bench Hard (BBH) (val) | Accuracy43.46 | 36 | |
| Reasoning | Big Bench Hard 3-shot | Accuracy41.46 | 18 | |
| Word Sorting | Big-Bench Hard Word Sorting | Success Rate79.8 | 4 | |
| Counting | Big-Bench Hard Counting | Success Rate91.9 | 4 | |
| Temporal Reasoning | BIG-bench Hard Temporal Sequences (test) | Test Accuracy62 | 4 | |
| Causal Reasoning | BIG-Bench Hard Causal Judgment (OOD) | Accuracy60.4 | 3 | |
| Spatial Reasoning | BIG-Bench Hard Navigate (OOD) | Accuracy57.6 | 3 |
Showing 7 of 7 rows