UKP-Athene: Multi-Sentence Textual Entailment for Claim Verification
About
The Fact Extraction and VERification (FEVER) shared task was launched to support the development of systems able to verify claims by extracting supporting or refuting facts from raw text. The shared task organizers provide a large-scale dataset for the consecutive steps involved in claim verification, in particular, document retrieval, fact extraction, and claim classification. In this paper, we present our claim verification pipeline approach, which, according to the preliminary results, scored third in the shared task, out of 23 competing systems. For the document retrieval, we implemented a new entity linking approach. In order to be able to rank candidate facts and classify a claim on the basis of several selected facts, we introduce two extensions to the Enhanced LSTM (ESIM).
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Fact Verification | FEVER (dev) | Label Accuracy68.49 | 57 | |
| Fact Verification | FEVER (test) | LA Score65.46 | 32 | |
| Fact Verification | FEVER 1.0 (dev) | Label Accuracy68.49 | 23 | |
| Fact Extraction and Verification | FEVER (test) | Label Accuracy (LA)65.22 | 18 | |
| Fact Verification | FEVER 1.0 (test) | Label Accuracy65.46 | 14 | |
| Fact Extraction and Verification | FEVER (dev) | Label Accuracy (LA)68.49 | 9 | |
| Fact Verification | FEVER (blind test) | Label Accuracy65.46 | 6 | |
| Retrieval | FEVER | Precision30.6 | 4 | |
| Document Retrieval | FEVER (dev) | OFEVER93.55 | 3 |