Knowledge Guided Text Retrieval and Reading for Open Domain Question Answering
About
We introduce an approach for open-domain question answering (QA) that retrieves and reads a passage graph, where vertices are passages of text and edges represent relationships that are derived from an external knowledge base or co-occurrence in the same article. Our goals are to boost coverage by using knowledge-guided retrieval to find more relevant passages than text-matching methods, and to improve accuracy by allowing for better knowledge-guided fusion of information across related passages. Our graph retrieval method expands a set of seed keyword-retrieved passages by traversing the graph structure of the knowledge base. Our reader extends a BERT-based architecture and updates passage representations by propagating information from related passages and their relations, instead of reading each passage in isolation. Experiments on three open-domain QA datasets, WebQuestions, Natural Questions and TriviaQA, show improved performance over non-graph baselines by 2-11% absolute. Our approach also matches or exceeds the state-of-the-art in every case, without using an expensive end-to-end training regime.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Open-domain Question Answering | TriviaQA (test) | Exact Match55.8 | 80 | |
| Open-domain Question Answering | TriviaQA open (test) | EM56 | 59 | |
| End-to-end Open-Domain Question Answering | NQ (test) | Exact Match (EM)34.7 | 50 | |
| Open-domain Question Answering | Natural Questions (NQ) | Exact Match (EM)34.7 | 46 | |
| End-to-end Open-Domain Question Answering | TriviaQA (test) | Exact Match (EM)56 | 40 | |
| Open-domain Question Answering | WebQuestions (WQ) Open-QA (test) | Exact Match36.4 | 38 | |
| Open-domain Question Answering | NaturalQ-Open (test) | EM31.8 | 37 | |
| Passage Reading | NQ | EM34.5 | 10 |