Natural Questions
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Open Question Answering | Natural Questions (NQ) (test) | Exact Match (EM)58.4 | 134 | |
| Retrieval Attack Defense | Natural Questions (NQ) | ASR0 | 99 | |
| Inference Efficiency | Natural Questions (NQ) | Relative Overhead (%)0.019 | 90 | |
| Open Domain Question Answering | Natural Questions (NQ) | Exact Match (EM)60.7 | 82 | |
| Question Answering | Natural Questions (NQ) (test) | Exact Match76 | 77 | |
| Over-refusal Evaluation | NQ (Natural Questions) | ORR0 | 72 | |
| Question Answering | Natural Questions (test) | EM61.65 | 72 | |
| Question Answering | NQ (Natural Questions) | EM78.3 | 70 | |
| RAG Attack Defense | Natural Questions | ASR0 | 63 | |
| Retrieval | Natural Questions (test) | Top-5 Recall92.1 | 62 | |
| Question Answering | NQ (Natural Questions) (test) | Accuracy68.6 | 60 | |
| Single-hop QA | NQ (Natural Questions) | EM72 | 52 | |
| Question Answering | Natural Questions | EM70.58 | 52 | |
| Question Answering | Natural Questions (NQ) | Accuracy49.3 | 48 | |
| Inference Cost Attack | Natural Questions (NQ) | wAA (%)100 | 45 | |
| Watermarking | Natural Questions (NQ) (test) | AUROC100 | 45 | |
| Question Answering | Natural Questions (NQ) (test) | Robust Accuracy68 | 45 | |
| Knowledge Evaluation | Natural Questions (NQ) (Evaluation) | Accuracy83 | 45 | |
| Passage retrieval | Natural Questions (NQ) (test) | Top-20 Accuracy85.2 | 45 | |
| Open-domain Question Answering | Natural Questions (NQ) (test) | Accuracy58.15 | 44 | |
| Embedding Alignment | Natural Questions (test) | Top-1 Accuracy100 | 40 | |
| Information Retrieval | Natural Questions | Recall@1100 | 40 | |
| RAG Poisoning Attack | Natural Questions | E2E-ASR64.08 | 36 | |
| Question Answering | Natural Questions | Accuracy46.39 | 36 | |
| Lexical Retrieval | Natural Questions | Recall@50.982 | 35 |
Showing 25 of 136 rows