SuperGPQA
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Scientific Reasoning | SuperGPQA | Mean@150.1 | 34 | |
| Knowledge-intensive reasoning | SuperGPQA | Overall Score63.4 | 31 | |
| Scientific Reasoning | SuperGPQA (test) | Pass@177.7 | 25 | |
| Reasoning | SuperGPQA | Length7,262 | 24 | |
| Reasoning | superGPQA | Accuracy (superGPQA)39.3 | 24 | |
| Code Reasoning | SuperGPQA Code SGPQA-1k | Accuracy47.4 | 24 | |
| Math Reasoning | SuperGPQA SGPQA-1k Math | Accuracy46.5 | 24 | |
| Multiple Choice Question Answering | SuperGPQA MCQA | Accuracy63.83 | 21 | |
| General Knowledge | SuperGPQA | pass@148.2 | 19 | |
| General Knowledge QA | superGPQA | Average Accuracy33.6 | 18 | |
| Reasoning | SuperGPQA | Pass@138.4 | 17 | |
| General Reasoning | SuperGPQA | Accuracy (General Reasoning)35.7 | 15 | |
| General Reasoning | SuperGPQA | Avg@k28.46 | 15 | |
| Question Answering | SuperGPQA (test) | Accuracy40.2 | 15 | |
| Scientific Reasoning | SuperGPQA | Accuracy44.7 | 15 | |
| Graduate-level Reasoning | SuperGPQA | Pass@136 | 14 | |
| LLM Routing | SUPERGPQA (val) | Top-1 Acc0.776 | 14 | |
| LLM Routing | SUPERGPQA | Top-1 Acc77.6 | 14 | |
| Scientific Question Answering | SuperGPQA* | Accuracy62.4 | 12 | |
| Science Question Answering | SuperGPQA | Avg@k28.46 | 12 | |
| Medical Question Answering | SuperGPQA Clinical | Accuracy35.71 | 12 | |
| Question Answering | SuperGPQA Law | Accuracy43.8 | 10 | |
| Question Answering | SuperGPQA | Pass@134.51 | 8 | |
| Science Question Answering | SuperGPQA | Score35.59 | 8 | |
| Medical Question Answering | SuperGPQA | Accuracy27.66 | 8 |
Showing 25 of 37 rows