NaturalQuestions
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Question Answering | NaturalQuestions filtered (dev) | EM44.3 | 48 | |
| Single-document retrieval | NaturalQuestions | F1 Score60.65 | 44 | |
| Open-Domain Question Answering | NaturalQuestions (NQ) | SubEM52.88 | 40 | |
| Question Answering | NaturalQuestions | EM66.59 | 39 | |
| Question Answering | NaturalQuestions 20 documents | Tokens Processed2,946 | 26 | |
| Information Retrieval | NaturalQuestions (NQ) (test) | Top-20 Acc82.24 | 23 | |
| Question Answering | NaturalQuestions processed | Accuracy83.05 | 22 | |
| Question Answering | NaturalQuestions Open | Exact Match7.7 | 12 | |
| Open-domain Question Answering | NaturalQuestions standard (test) | Accuracy45.5 | 12 | |
| Single-document retrieval | NaturalQuestions | Latency (s)0.0047 | 11 | |
| Long document retrieval | NaturalQuestions (test) | F1 Score59.98 | 11 | |
| Open-Domain Question Answering | NaturalQuestions (NQ) v1.0 (test) | Acc@2083 | 11 | |
| Knowledge-intensive QA | NaturalQuestions (NQ) 5-shot | EM Accuracy42.85 | 10 | |
| Open-domain QA | NaturalQuestions (NQ) top 1000 samples (test) | Exact Match40.3 | 10 | |
| Open-Domain Question Answering | NaturalQuestions (test) | Top-1 EM54.4 | 9 | |
| Closed-book Question Answering | NaturalQuestions (test) | EM23 | 9 | |
| Single-hop QA Retrieval | NaturalQuestions (NQ) (test) | R@280.8 | 8 | |
| Question Answering | NaturalQuestions (val) | Exact Match5.7 | 5 | |
| Language | NaturalQuestions | Accuracy0.433 | 3 |
Showing 19 of 19 rows