Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Reinforce to Learn, Elect to Reason: A Dual Paradigm for Video Reasoning

About

Video reasoning has advanced with large multimodal models (LMMs), yet their inference is often a single pass that returns an answer without verifying whether the reasoning is evidence-aligned. We introduce Reinforce to Learn, Elect to Reason (RLER), a dual paradigm that decouples learning to produce evidence from obtaining a reliable answer. In RLER-Training, we optimize the policy with group-relative reinforcement learning (RL) and 3 novel task-driven rewards: Frame-sensitive reward grounds reasoning on explicit key frames, Think-transparency reward shapes readable and parsable reasoning traces, and Anti-repetition reward boosts information density. These signals teach the model to emit structured, machine-checkable evidence and potentiate reasoning capabilities. In RLER-Inference, we apply a train-free orchestrator that generates a small set of diverse candidates, parses their answers and cited frames, scores them by evidence consistency, confidence, transparency, and non-redundancy, and then performs a robust evidence-weighted election. This closes the loop between producing and using evidence, improving reliability and interpretability without enlarging the model. We comprehensively evaluate RLER against various open-source and RL-based LMMs on 8 representative benchmarks. RLER achieves state of the art across all benchmarks and delivers an average improvement of 6.3\% over base models, while using on average 3.1 candidates per question, indicating a favorable balance between compute and quality. The results support a simple thesis: making evidence explicit during learning and electing by evidence during inference is a robust path to trustworthy video reasoning.

Songyuan Yang, Weijiang Yu, Jilin Ma, Ziyu Liu, Guijian Tang, Wenjing Yang, Huibin Tan, Nong Xiao• 2026

Related benchmarks

TaskDatasetResultRank
Video UnderstandingMVBench
Accuracy72.9
425
Long Video UnderstandingLVBench
Accuracy50.7
133
Temporal Video UnderstandingTempCompass--
68
Video UnderstandingVideoMME--
60
Video ReasoningVideo-MMMU
Accuracy54.2
45
Long Video UnderstandingLongVideoBench
Accuracy63
23
Video ReasoningVSIBench
Accuracy43.3
10
General Video UnderstandingWildVideo
Accuracy57.5
5
Showing 8 of 8 rows

Other info

Follow for update