QuALITY
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Question Answering | QuALITY (test) | Accuracy77.71 | 90 | |
| Multiple-Choice Question Answering | QuALITY | Accuracy56.997 | 19 | |
| Question Answering | QUALITY | Exact Match83.8 | 10 | |
| Multiple-Choice Question Answering | QuALITY Hard Subset | Accuracy62.9 | 6 | |
| Multiple-Choice Question Answering | QuALITY (test) | Accuracy74.7 | 6 | |
| Reading Comprehension | QuaLITY | P@176.4 | 6 | |
| Long-context Question Answering | QuALITY | Accuracy73.63 | 5 | |
| Question Answering | QuALITY multiple choice (test) | Accuracy86.09 | 4 | |
| Reading Comprehension | QuALITY 0-shot | Accuracy40.9 | 4 | |
| Question Answering | QuALITY ZeroSCROLLS leaderboard (test) | Accuracy72.8 | 4 | |
| Question Answering | QuALITY hard | Accuracy76.2 | 4 | |
| Reading Comprehension | QuALITY (test) | Accuracy87.6 | 3 | |
| Question Answering | QuALITY 0-shot | Log Accuracy38.9 | 2 | |
| Question Answering | QuALITY (dev) | Exact Match37.6 | 2 |
Showing 14 of 14 rows