UniK-QA: Unified Representations of Structured and Unstructured Knowledge for Open-Domain Question Answering
About
We study open-domain question answering with structured, unstructured and semi-structured knowledge sources, including text, tables, lists and knowledge bases. Departing from prior work, we propose a unifying approach that homogenizes all sources by reducing them to text and applies the retriever-reader model which has so far been limited to text sources only. Our approach greatly improves the results on knowledge-base QA tasks by 11 points, compared to latest graph-based methods. More importantly, we demonstrate that our unified knowledge (UniK-QA) model is a simple and yet effective way to combine heterogeneous sources of knowledge, advancing the state-of-the-art results on two popular question answering benchmarks, NaturalQuestions and WebQuestions, by 3.5 and 2.6 points, respectively. The code of UniK-QA is available at: https://github.com/facebookresearch/UniK-QA.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Knowledge Base Question Answering | WEBQSP (test) | Hit@179.1 | 143 | |
| Open Question Answering | Natural Questions (NQ) (test) | Exact Match (EM)54.6 | 134 | |
| Open-domain Question Answering | TriviaQA (test) | Exact Match65.1 | 80 | |
| Open-domain Question Answering | WebQuestions (WebQ) (test) | Exact Match (EM)57.8 | 55 | |
| End-to-end Open-Domain Question Answering | NQ (test) | Exact Match (EM)54 | 50 | |
| Knowledge Base Question Answering | WebQSP Freebase (test) | F1 Score79.1 | 46 | |
| Open-domain Question Answering | CuratedTREC (test) | Exact Match (EM)55.3 | 26 | |
| Temporal Question Answering | TIME QUESTIONS 1.0 (test) | P@142.4 | 18 | |
| Temporal Question Answering | TIQ 1.0 (test) | P@10.425 | 10 | |
| Open-domain Question Answering | COMPMIX (test) | Exact Match44 | 9 |