Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Faithful Temporal Question Answering over Heterogeneous Sources

About

Temporal question answering (QA) involves time constraints, with phrases such as "... in 2019" or "... before COVID". In the former, time is an explicit condition, in the latter it is implicit. State-of-the-art methods have limitations along three dimensions. First, with neural inference, time constraints are merely soft-matched, giving room to invalid or inexplicable answers. Second, questions with implicit time are poorly supported. Third, answers come from a single source: either a knowledge base (KB) or a text corpus. We propose a temporal QA system that addresses these shortcomings. First, it enforces temporal constraints for faithful answering with tangible evidence. Second, it properly handles implicit questions. Third, it operates over heterogeneous sources, covering KB, text and web tables in a unified manner. The method has three stages: (i) understanding the question and its temporal conditions, (ii) retrieving evidence from all sources, and (iii) faithfully answering the question. As implicit questions are sparse in prior benchmarks, we introduce a principled method for generating diverse questions. Experiments show superior performance over a suite of baselines.

Zhen Jia, Philipp Christmann, Gerhard Weikum• 2024

Related benchmarks

TaskDatasetResultRank
Temporal Question AnsweringTIME QUESTIONS 1.0 (test)
P@157.1
18
Temporal Question AnsweringTIQ 1.0 (test)
P@10.491
10
Fact VerificationT-FEVER
Macro F1 (Overall)49.03
7
Fact VerificationT-FEVEROUS
Macro F138.63
7
Fact VerificationT-QuanTemp
Macro F149.65
7
Temporal Claim VerificationT-FEVER
Micro F1 (Overall)51.75
7
Temporal Claim VerificationT-FEVEROUS
Micro F1 (Overall)51.51
7
Temporal Claim VerificationT-QuanTemp
Micro F159.11
7
Temporal Fact VerificationChronoClaims
Overall Macro F160.84
7
Temporal Claim VerificationChronoClaims
Micro F1 (Overall)61.08
7
Showing 10 of 10 rows

Other info

Code

Follow for update