Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Entropy Is Not Enough: Unlocking Effective Reinforcement Learning for Visual Reasoning via Vision-Anchored Token Selection

About

While token-level entropy is commonly recognized as effective for credit assignment in text-only reinforcement learning with verifiable rewards (RLVR), it remains unclear whether this mechanism still holds in visual reasoning. Our controlled study shows that this mechanism collapses in visual reasoning due to the omission of vision-sensitive tokens with naturally low entropy. Although existing multimodal RL methods increasingly acknowledge the importance of visual perception, they struggle to satisfy the inherent demand for interleaving precise perceptual grounding with semantic reasoning, either lacking systematic visual measurements or overlooking that token entropy primarily drives semantic exploration. To address this, we introduce VEPO (Vision-Entropy token-selection for Policy Optimization), an effective RL framework explicitly integrating visual sensitivity with token entropy via a principled multiplicative coupling, where VEPO redirects gradient credit toward tokens which are simultaneously visually grounded and highly informative. Extensive experiments demonstrate VEPO's leading performance, significantly outperforming the entropy-only baseline by 2.28 points at 7B-scale and 3.15 points at 3B-scale. Ablations further substantiate the soundness of our method.

Senjie Jin, Peixin Wang, Boyang Liu, Xiaoran Fan, Shuo Li, Zhiheng Xi, Jiazheng Zhang, Yuhao Zhou, Tao Gui, Qi Zhang, Xuanjing Huang• 2026

Related benchmarks

TaskDatasetResultRank
Visual Mathematical ReasoningMathVista
Accuracy72
448
Visual Mathematical ReasoningMathVision
Accuracy28.31
298
Visual Mathematical ReasoningMathVerse
Accuracy48.93
194
Visual Mathematical ReasoningWeMath
Accuracy69.54
193
Multimodal Math ReasoningMMK12
Accuracy69.64
41
Visual PerceptionHallusionBench
Accuracy60.46
24
Geometry reasoningGeometry3K (val)
Accuracy44.93
21
Visual Math ReasoningGeo3K
Accuracy51.58
9
Visual Mathematical ReasoningMMK12 (val)
Accuracy58.86
9
Showing 9 of 9 rows

Other info

Follow for update