Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

Do MLLMs Really See It: Reinforcing Visual Attention in Multimodal LLMs

About

While chain-of-thought (CoT) reasoning has substantially improved multimodal large language models (MLLMs) on complex reasoning tasks, existing approaches largely rely on long textual reasoning trajectories and provide limited mechanisms for learning stable visual attention policies. Our analysis shows that current MLLMs exhibit weak visual focus: early-stage visual misalignment is rarely corrected during subsequent reasoning, leading to error propagation and failed inferences. We argue that this limitation stems from inadequate credit assignment for visual attention during training. To address this issue, we propose SAYO, a visual reasoning model trained with a reinforcement learning (RL) framework that introduces a region-level visual attention-based reward. This reward explicitly aligns optimization signals with visually grounded reasoning steps, enabling the model to learn more reliable attention behaviors. Extensive experiments across multiple multimodal benchmarks demonstrate that SAYO consistently improves performance on diverse reasoning and perception tasks.

Siqu Ou, Tianrui Wan, Zhiyuan Zhao, Junyu Gao, Xuelong Li• 2026

Related benchmarks

TaskDatasetResultRank
Diagram UnderstandingAI2D
Accuracy83.06
167
Chart UnderstandingChartQA
Accuracy82.28
83
Mathematical ReasoningWeMath
Accuracy64.83
75
Visual ReasoningV*Bench
Accuracy83.25
58
Mathematical ReasoningMATH-Vision
Accuracy25.26
32
General Visual ReasoningMMStar
Accuracy65.27
29
General Visual ReasoningMME-RealWorld-Lite
Accuracy62.85
17
General Visual ReasoningM3CoT
Accuracy68.46
17
Scientific Figure ReasoningCharXiv
Accuracy43.2
17
Showing 9 of 9 rows

Other info

Follow for update