OBQA
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Question Answering | OBQA | Accuracy94.95 | 347 | |
| Commonsense Reasoning | OBQA | Accuracy89.2 | 187 | |
| Question Answering | OBQA (test) | Accuracy92.7 | 90 | |
| Multiple Choice Question Answering | OBQA | Accuracy93.2 | 79 | |
| Reasoning | OBQA | Accuracy97.67 | 46 | |
| General Knowledge QA | OBQA | Normalized Accuracy48 | 44 | |
| General capability | OBQA (test) | Normalized Accuracy44 | 42 | |
| Out-of-Distribution Detection | OBQA to MMLU | AUROC87.09 | 41 | |
| Question Answering | OBQA | Zero-shot Accuracy35.2 | 36 | |
| OpenBook Question Answering | OBQA | Accuracy44.2 | 32 | |
| Question Answering | OBQA | Accuracy (Normalized)41.8 | 29 | |
| Uncertainty Estimation | OBQA | AUROC88.03 | 24 | |
| Commonsense Reasoning | OBQA | First-Token Accuracy91.4 | 24 | |
| Multiple Choice Question Answering | OBQA (test) | Accuracy71.6 | 21 | |
| Commonsense Question Answering | OBQA | Accuracy93.4 | 19 | |
| Zero-shot Prediction | OBQA | Accuracy31.4 | 17 | |
| Multiple Choice Question Answering | OBQA (dev) | Accuracy86.1 | 17 | |
| Question Answering | OBQA | Accuracy86 | 16 | |
| Speech-to-Text Question-Answering | OBQA | Accuracy83.08 | 16 | |
| Question Answering | OBQA | Accuracy88.8 | 14 | |
| Question Answering | OBQA | Accuracy95.2 | 12 | |
| Common-sense reasoning | OBQA In-Distribution | Accuracy88.43 | 12 | |
| Reasoning | OBQA (leave-one-out setup) | Average Accuracy87.7 | 12 | |
| Question Answering | OBQA | Accuracy90.1 | 12 | |
| Question Answering | OBQA (out-of-domain) | Acc95.59 | 12 |
Showing 25 of 42 rows