DecAF: Joint Decoding of Answers and Logical Forms for Question Answering over Knowledge Bases
About
Question answering over knowledge bases (KBs) aims to answer natural language questions with factual information such as entities and relations in KBs. Previous methods either generate logical forms that can be executed over KBs to obtain final answers or predict answers directly. Empirical results show that the former often produces more accurate answers, but it suffers from non-execution issues due to potential syntactic and semantic errors in the generated logical forms. In this work, we propose a novel framework DecAF that jointly generates both logical forms and direct answers, and then combines the merits of them to get the final answers. Moreover, different from most of the previous methods, DecAF is based on simple free-text retrieval without relying on any entity linking tools -- this simplification eases its adaptation to different datasets. DecAF achieves new state-of-the-art accuracy on WebQSP, FreebaseQA, and GrailQA benchmarks, while getting competitive results on the ComplexWebQuestions benchmark.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Knowledge Base Question Answering | WEBQSP (test) | Hit@182.1 | 143 | |
| Knowledge Graph Question Answering | CWQ (test) | Hits@170.4 | 69 | |
| Multi-hop Knowledge Graph Question Answering | WebQSP | Hits@182.1 | 50 | |
| Knowledge Base Question Answering | WebQSP Freebase (test) | F1 Score78.8 | 46 | |
| Multi-hop Knowledge Graph Question Answering | CWQ | Hits@170.4 | 46 | |
| Knowledge Base Question Answering | CWQ (test) | F1 Score70.4 | 42 | |
| Knowledge Base Question Answering | GrailQA v1.0 (test) | Overall EM68.4 | 33 | |
| Knowledge Base Question Answering | CWQ Freebase (test) | Hits@170.4 | 19 | |
| Knowledge Base Question Answering | WebQSP v1 (test) | F1 Score78.8 | 16 | |
| Knowledge Base Question Answering | GrailQA v1.0 (dev) | F181.4 | 9 |