Natural Questions
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Open Question Answering | Natural Questions (NQ) (test) | Exact Match (EM)58.4 | 134 | |
| Retrieval Attack Defense | Natural Questions (NQ) | ASR0 | 99 | |
| Inference Efficiency | Natural Questions (NQ) | Relative Overhead (%)0.019 | 90 | |
| Open Domain Question Answering | Natural Questions (NQ) | Exact Match (EM)60.7 | 82 | |
| Question Answering | Natural Questions (NQ) (test) | Exact Match76 | 77 | |
| Over-refusal Evaluation | NQ (Natural Questions) | ORR0 | 72 | |
| Question Answering | Natural Questions (test) | EM61.65 | 72 | |
| Question Answering | NQ (Natural Questions) | EM78.3 | 70 | |
| RAG Attack Defense | Natural Questions | ASR0 | 63 | |
| Retrieval | Natural Questions (test) | Top-5 Recall92.1 | 62 | |
| Question Answering | NQ (Natural Questions) (test) | Accuracy68.6 | 60 | |
| Single-hop QA | NQ (Natural Questions) | EM72 | 52 | |
| Question Answering | Natural Questions | EM70.58 | 52 | |
| Question Answering | Natural Questions (NQ) | Accuracy49.3 | 48 | |
| Watermarking | Natural Questions (NQ) (test) | AUROC100 | 45 | |
| Question Answering | Natural Questions (NQ) (test) | Robust Accuracy68 | 45 | |
| Knowledge Evaluation | Natural Questions (NQ) (Evaluation) | Accuracy83 | 45 | |
| Passage retrieval | Natural Questions (NQ) (test) | Top-20 Accuracy85.2 | 45 | |
| Embedding Alignment | Natural Questions (test) | Top-1 Accuracy100 | 40 | |
| Question Answering | Natural Questions | Accuracy46.39 | 36 | |
| Open-QA Evaluation | EVOUNA-NaturalQuestions | F1 Score97.9 | 35 | |
| Honesty Alignment | Natural Questions (NQ) In-Domain | AUROC85.16 | 33 | |
| Single-hop Question Answering | Natural Questions (NQ) (test) | EM47.5 | 33 | |
| Open-Domain Question Answering | NQ (Natural Questions) | EM51.4 | 33 | |
| Question Answering | Natural Questions (NQ) | Exact Match (EM)45.68 | 32 |
Showing 25 of 122 rows