PopQA
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Single-Hop Question Answering | PopQA | EM73.6 | 200 | |
| Question Answering | PopQA | Accuracy68.4 | 186 | |
| Question Answering | PopQA | Accuracy87.12 | 158 | |
| Question Answering | PopQA | Exact Match64.2 | 133 | |
| Question Answering | PopQA (test) | Accuracy77.2 | 122 | |
| Question Answering | PopQA | EM56.6 | 112 | |
| Hallucination Detection | PopQA | AUC96.18 | 97 | |
| Question Answering | PopQA | Score52.7 | 86 | |
| Uncertainty Quantification | PopQA 500 randomly sampled queries (test) | AUROC0.8709 | 70 | |
| General QA | PopQA | Exact Match (EM)52 | 58 | |
| Factual Knowledge Evaluation | PopQA | Accuracy35.3 | 56 | |
| General Question Answering | PopQA | EM45.2 | 51 | |
| Knowledge Retrieval | PopQA | F1 Score67.85 | 45 | |
| Single-hop Question Answering | PopQA (test) | Accuracy51.5 | 43 | |
| Knowledge-conflict Question Answering | PopQA CF-RAG | CF92.54 | 36 | |
| Simple Question Answering | PopQA (test) | RAG F165.24 | 36 | |
| Inter-Context Knowledge Conflict | popQA | ACCL90.28 | 32 | |
| Question Answering | PopQA | F1 Score59.9 | 30 | |
| Question Answering | PopQA | EM (%)47.82 | 27 | |
| Question Answering | PopQA | EM46.1 | 27 | |
| General Question Answering | PopQA (test) | EM49.3 | 27 | |
| Simple Question Answering | PopQA | F156.3 | 26 | |
| Question Answering | PopQA | Accuracy (Acc)70.7 | 26 | |
| Abstention | PopQA (test) | AUARC66.06 | 25 | |
| Abstention | PopQA | Abstain Accuracy81.6 | 25 |
Showing 25 of 131 rows