Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

TARAC: Mitigating Hallucination in LVLMs via Temporal Attention Real-time Accumulative Connection

About

Large Vision-Language Models have demonstrated remarkable capabilities, yet they suffer from hallucinations that limit practical deployment. While various mitigation strategies exist, they often incur high computational overhead or require extensive retraining. In this paper, we address the issue of visual attention decay during generation, a key factor contributing to hallucinations. We propose Temporal Attention Real-time Accumulative Connection (TARAC), a novel training-free framework that dynamically accumulates and re-injects historical attention to sustain visual grounding. Inspired by cognitive reinforcement mechanisms, TARAC operates as a lightweight, plug-and-play module. Extensive experiments across diverse models (e.g., LLaVA, Qwen2-VL) and benchmarks demonstrate that TARAC significantly outperforms state-of-the-art methods. Remarkably, it achieves these gains with negligible inference overhead ($\sim$4\% TPOT increase), compared to the substantial costs of existing training-free baselines. Specifically, TARAC reduces hallucinated sentences by 25.2\% on CHAIR and improves Perception score by +10.65 on MME, validating its effectiveness and efficiency.

Lei Jiang, Chunzhao Xie, Tongxuan Liu, Yuting Zeng, jinrong Guo, Yunheng Shen, Weizhe Huang, Jing Li, Xiaohua Xu• 2025

Related benchmarks

TaskDatasetResultRank
Object HallucinationPOPE (Random)
F1 Score89.01
285
Object HallucinationPOPE Popular
F1 Score87.22
273
Hallucination EvaluationAMBER
CHAIR6.7
172
Hallucination assessmentAMBER
CHAIR_s5
56
Object Hallucination EvaluationPOPE Adversarial
Accuracy84.72
55
Image CaptioningChair (test)
Cs Score43
22
Perception EvaluationMME Perception
Score1.71e+3
21
Text Fluency EvaluationAMBER
PPL113.13
9
Hallucination MitigationSHR (test)
SPI4.93
9
Object Hallucination AssessmentChair (test)
CS (%)30
9
Showing 10 of 10 rows

Other info

Follow for update