ARC
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Question Answering | ARC Challenge | Accuracy96.3 | 749 | |
| Question Answering | ARC Easy | Accuracy98.2 | 386 | |
| Question Answering | ARC Easy | Normalized Acc96.4 | 385 | |
| Question Answering | ARC-E | Accuracy95.23 | 242 | |
| Science Question Answering | ARC Challenge | Accuracy96 | 234 | |
| Reasoning | ARC Easy | Accuracy96.63 | 183 | |
| Question Answering | ARC-C | Accuracy94.1 | 166 | |
| Question Answering | ARC | Accuracy94.6 | 154 | |
| Science Question Answering | ARC-E | Accuracy97.53 | 138 | |
| Commonsense Reasoning | ARC Challenge | Accuracy93.28 | 132 | |
| Science Question Answering | ARC-C | Accuracy96.3 | 127 | |
| Multiple Choice Question Answering | ARC Easy | Accuracy84.3 | 122 | |
| Multiple-choice Question Answering | ARC Challenge | Acc74.7 | 106 | |
| Science Question Answering | ARC Easy | Accuracy96.95 | 101 | |
| Reasoning | ARC | Accuracy92.34 | 83 | |
| Question Answering | ARC Challenge (val) | Accuracy89.6 | 72 | |
| Reasoning | ARC Challenge | Accuracy91.09 | 70 | |
| Question Answering | ARC-C | Accuracy0.71 | 68 | |
| Question Answering | ARC (test) | Accuracy90.5 | 67 | |
| Question Answering | ARC Challenge (test) | Accuracy64.25 | 63 | |
| Zero-shot Accuracy | ARC Easy | Zero-shot Acc (ARC Easy)78.2 | 63 | |
| Commonsense Reasoning | ARC-E | Accuracy96.4 | 62 | |
| Scientific Reasoning | ARC Challenge | Accuracy92.5 | 56 | |
| Common-sense reasoning | ARC easy | ARC (easy) Accuracy67.97 | 52 | |
| Commonsense Reasoning | ARC-C | Accuracy91.1 | 51 |
Showing 25 of 182 rows
...