JustiLM: Few-shot Justification Generation for Explainable Fact-Checking of Real-world Claims
About
Justification is an explanation that supports the veracity assigned to a claim in fact-checking. However, the task of justification generation is previously oversimplified as summarization of fact-check article authored by fact-checkers. Therefore, we propose a realistic approach to generate justification based on retrieved evidence. We present a new benchmark dataset called ExClaim for \underline{Ex}plainable fact-checking of real-world \underline{Claim}s, and introduce JustiLM, a novel few-shot \underline{Justi}fication generation based on retrieval-augmented \underline{L}anguage \underline{M}odel by using fact-check articles as auxiliary resource during training only. Experiments show that JustiLM achieves promising performance in justification generation compared to strong baselines, and can also enhance veracity classification with a straightforward extension.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Claim Verification | ChartCheck | Macro F10.632 | 38 | |
| Claim Verification | AIChartClaim | Macro F169.4 | 38 | |
| Claim Verification | MR2 | Macro F172.5 | 32 | |
| Claim Verification | Mocheg | Macro F145 | 32 | |
| Explanation Generation | AIChartClaim 1.0 (test) | ROUGE-126.8 | 9 | |
| Explanation Generation | ChartCheck 1.0 (test) | ROUGE-141.1 | 9 | |
| Explanation Generation | AIChartClaim | ROUGE-L21.7 | 9 | |
| Explanation Generation | ChartCheck | ROUGE-L34.7 | 9 | |
| Explanation Generation | AIChartClaim (test) | ROUGE-131.4 | 9 | |
| Explanation Generation | ChartCheck (test) | ROUGE-140 | 9 |