Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Video-Thinker: Sparking "Thinking with Videos" via Reinforcement Learning

About

Recent advances in image reasoning methods, particularly "Thinking with Images", have demonstrated remarkable success in Multimodal Large Language Models (MLLMs); however, this dynamic reasoning paradigm has not yet been extended to video reasoning tasks. In this paper, we propose Video-Thinker, which empowers MLLMs to think with videos by autonomously leveraging their intrinsic "grounding" and "captioning" capabilities to generate reasoning clues throughout the inference process. To spark this capability, we construct Video-Thinker-10K, a curated dataset featuring autonomous tool usage within chain-of-thought reasoning sequences. Our training strategy begins with Supervised Fine-Tuning (SFT) to learn the reasoning format, followed by Group Relative Policy Optimization (GRPO) to strengthen this reasoning capability. Through this approach, Video-Thinker enables MLLMs to autonomously navigate grounding and captioning tasks for video reasoning, eliminating the need for constructing and calling external tools. Extensive experiments demonstrate that Video-Thinker achieves significant performance gains on both in-domain tasks and challenging out-of-domain video reasoning benchmarks, including Video-Holmes, CG-Bench-Reasoning, and VRBench. Our Video-Thinker-7B substantially outperforms existing baselines such as Video-R1 and establishes state-of-the-art performance among 7B-sized MLLMs.

Shijian Wang, Jiarui Jin, Xingjian Wang, Linxin Song, Runhao Fu, Hecheng Wang, Zongyuan Ge, Yuan Lu, Xuelian Cheng• 2025

Related benchmarks

TaskDatasetResultRank
Video UnderstandingMVBench
Accuracy58.9
563
Long Video UnderstandingLVBench
Accuracy54.3
218
Temporal Video UnderstandingTempCompass
Accuracy67.5
141
Video UnderstandingLongVideoBench--
123
Video UnderstandingMLVU
Accuracy65.2
114
Video ReasoningVideo-Holmes
Accuracy43.1
83
General Video UnderstandingVideo-MME
Accuracy60
82
Video UnderstandingMMVU
Accuracy64.5
76
Video ReasoningVideo-MMMU
Accuracy43.9
68
Temporal GroundingCharades-STA (test)--
68
Showing 10 of 30 rows

Other info

Follow for update