Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

RetinaQA: A Robust Knowledge Base Question Answering Model for both Answerable and Unanswerable Questions

About

An essential requirement for a real-world Knowledge Base Question Answering (KBQA) system is the ability to detect the answerability of questions when generating logical forms. However, state-of-the-art KBQA models assume all questions to be answerable. Recent research has found that such models, when superficially adapted to detect answerability, struggle to satisfactorily identify the different categories of unanswerable questions, and simultaneously preserve good performance for answerable questions. Towards addressing this issue, we propose RetinaQA, a new KBQA model that unifies two key ideas in a single KBQA architecture: (a) discrimination over candidate logical forms, rather than generating these, for handling schema-related unanswerability, and (b) sketch-filling-based construction of candidate logical forms for handling data-related unaswerability. Our results show that RetinaQA significantly outperforms adaptations of state-of-the-art KBQA models in handling both answerable and unanswerable questions and demonstrates robustness across all categories of unanswerability. Notably, RetinaQA also sets a new state-of-the-art for answerable KBQA, surpassing existing models.

Prayushi Faldu, Indrajit Bhattacharya, Mausam• 2024

Related benchmarks

TaskDatasetResultRank
Knowledge Base Question AnsweringWEBQSP (test)--
143
Knowledge Base Question AnsweringGrailQA v1.0 (test)
Overall EM77.45
33
Knowledge Base Question AnsweringGrailQAbility answerable 1.0 (test)
F1 (L)81.22
27
Knowledge Base Question AnsweringGrailQAbility unanswerable 1.0 (test)
F1 (L)94.21
19
Knowledge Base Question AnsweringGrailQAbility answerable (IID)
F1(L)89
8
Knowledge Base Question AnsweringGrailQAbility answerable zero-shot
F1 (L)74.84
8
Knowledge Base Question AnsweringGrailQA (val)
Overall F183.33
4
Knowledge Base Question AnsweringGrailQAbility unanswerable questions (test IID)
F1(R)97.01
4
Knowledge Base Question AnsweringGrailQAbility unanswerable questions Zero-Shot (test)
F1(R)88.31
4
Showing 9 of 9 rows

Other info

Code

Follow for update