Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

TELLER: A Trustworthy Framework for Explainable, Generalizable and Controllable Fake News Detection

About

The proliferation of fake news has emerged as a severe societal problem, raising significant interest from industry and academia. While existing deep-learning based methods have made progress in detecting fake news accurately, their reliability may be compromised caused by the non-transparent reasoning processes, poor generalization abilities and inherent risks of integration with large language models (LLMs). To address this challenge, we propose {\methodname}, a novel framework for trustworthy fake news detection that prioritizes explainability, generalizability and controllability of models. This is achieved via a dual-system framework that integrates cognition and decision systems, adhering to the principles above. The cognition system harnesses human expertise to generate logical predicates, which guide LLMs in generating human-readable logic atoms. Meanwhile, the decision system deduces generalizable logic rules to aggregate these atoms, enabling the identification of the truthfulness of the input news across diverse domains and enhancing transparency in the decision-making process. Finally, we present comprehensive evaluation results on four datasets, demonstrating the feasibility and trustworthiness of our proposed framework. Our implementation is available at \url{https://github.com/less-and-less-bugs/Trust_TELLER}.

Hui Liu, Wenya Wang, Haoru Li, Haoliang Li• 2024

Related benchmarks

TaskDatasetResultRank
Fake News DetectionPolitiFact
Accuracy83.82
53
Binary ClassificationLIAR Closed
Accuracy79.15
23
Multi-ClassificationLIAR Open
Accuracy38.63
23
Binary ClassificationLIAR Open
Accuracy84.21
22
Multi-ClassificationLIAR Closed
Accuracy26.99
22
Fake News DetectionConstraint
Accuracy87.78
15
Fake News DetectionGossipCop CP -> G
Accuracy70.93
8
Fake News DetectionConstraint GP -> C
Accuracy85.09
8
Fake News DetectionPolitiFact CG -> P
Accuracy82.82
8
Binary ClassificationLIAR (test)
Accuracy67.73
3
Showing 10 of 16 rows

Other info

Code

Follow for update