Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

TARAC: Mitigating Hallucination in LVLMs via Temporal Attention Real-time Accumulative Connection

About

Large Vision-Language Models have demonstrated remarkable capabilities, yet they suffer from hallucinations that limit practical deployment. While various mitigation strategies exist, they often incur high computational overhead or require extensive retraining. In this paper, we address the issue of visual attention decay during generation, a key factor contributing to hallucinations. We propose Temporal Attention Real-time Accumulative Connection (TARAC), a novel training-free framework that dynamically accumulates and re-injects historical attention to sustain visual grounding. Inspired by cognitive reinforcement mechanisms, TARAC operates as a lightweight, plug-and-play module. Extensive experiments across diverse models (e.g., LLaVA, Qwen2-VL) and benchmarks demonstrate that TARAC significantly outperforms state-of-the-art methods. Remarkably, it achieves these gains with negligible inference overhead ($\sim$4\% TPOT increase), compared to the substantial costs of existing training-free baselines. Specifically, TARAC reduces hallucinated sentences by 25.2\% on CHAIR and improves Perception score by +10.65 on MME, validating its effectiveness and efficiency.

Lei Jiang, Chunzhao Xie, Tongxuan Liu, Yuting Zeng, jinrong Guo, Yunheng Shen, Weizhe Huang, Jing Li, Xiaohua Xu• 2025

Related benchmarks

TaskDatasetResultRank
Object HallucinationPOPE Popular
Accuracy87.88
406
Object HallucinationPOPE (Random)
F1 Score89.01
338
Hallucination EvaluationAMBER
CHAIR6.7
267
Object Hallucination EvaluationPOPE Adversarial
Accuracy84.72
174
Hallucination assessmentAMBER
CHAIR_s5
56
Image CaptioningChair (test)
Cs Score43
22
Object Hallucination AssessmentChair (test)
CS (%)30
22
Perception EvaluationMME Perception
Score1.71e+3
21
Text Fluency EvaluationAMBER
PPL113.13
9
Hallucination MitigationSHR (test)
SPI4.93
9
Showing 10 of 10 rows

Other info

Follow for update