Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

FaithLens: Detecting and Explaining Faithfulness Hallucination

About

Recognizing whether outputs from large language models (LLMs) contain faithfulness hallucination is crucial for real-world applications, e.g., retrieval-augmented generation and summarization. In this paper, we introduce FaithLens, a cost-efficient and effective faithfulness hallucination detection model that can jointly provide binary predictions and corresponding explanations to improve trustworthiness. To achieve this, we first synthesize training data with explanations via advanced LLMs and apply a well-defined data filtering strategy to ensure label correctness, explanation quality, and data diversity. Subsequently, we fine-tune the model on these well-curated training data as a cold start and further optimize it with rule-based reinforcement learning, using rewards for both prediction correctness and explanation quality. Results on 12 diverse tasks show that the 8B-parameter FaithLens outperforms advanced models such as GPT-5.2 and o3. Also, FaithLens can produce high-quality explanations, delivering a distinctive balance of trustworthiness, efficiency, and effectiveness.

Shuzheng Si, Qingyi Wang, Haozhe Zhao, Yuzhuo Bai, Guanqiao Chen, Kangyang Luo, Gang Chen, Fanchao Qi, Minjia Zhang, Baobao Chang, Maosong Sun• 2025

Related benchmarks

TaskDatasetResultRank
Faithfulness Hallucination DetectionLLM-AggreFact & HoVer Refined
Overall Std Dev4.6
14
Multi-hop Faithfulness Hallucination DetectionHoVer Refined
Macro F182.9
14
Faithfulness Hallucination DetectionLLM-AggreFact Refined
Agg-CNN84.9
14
Faithfulness DetectionFCGPT
Accuracy92.4
10
Faithfulness DetectionRAGTruth
Accuracy85.9
10
Faithfulness DetectionFaithCoT-Bench
F1 Score37.6
10
Faithfulness DetectionStep-level Benchmark In-domain Math
FF161.2
10
Faithfulness DetectionIn-domain Step-level Benchmark Reasoning
FF171.2
10
Faithfulness DetectionIn-domain Step-level Benchmark Knowledge
FF169.7
10
Faithfulness DetectionIn-domain Step-level Benchmark Agent
FF153.3
10
Showing 10 of 11 rows

Other info

GitHub

Follow for update