Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

MedScope: Incentivizing "Think with Videos" for Clinical Reasoning via Coarse-to-Fine Tool Calling

About

Long-form clinical videos are central to visual evidence-based decision-making, with growing importance for applications such as surgical robotics and related settings. However, current multimodal large language models typically process videos with passive sampling or weakly grounded inspection, which limits their ability to iteratively locate, verify, and justify predictions with temporally targeted evidence. To close this gap, we propose MedScope, a tool-using clinical video reasoning model that performs coarse-to-fine evidence seeking over long-form procedures. By interleaving intermediate reasoning with targeted tool calls and verification on retrieved observations, MedScope produces more accurate and trustworthy predictions that are explicitly grounded in temporally localized visual evidence. To address the lack of high-fidelity supervision, we build ClinVideoSuite, an evidence-centric, fine-grained clinical video suite. We then optimize MedScope with Grounding-Aware Group Relative Policy Optimization (GA-GRPO), which directly reinforces tool use with grounding-aligned rewards and evidence-weighted advantages. On full and fine-grained video understanding benchmarks, MedScope achieves state-of-the-art performance in both in-domain and out-of-domain evaluations. Our approach illuminates a path toward medical AI agents that can genuinely "think with videos" through tool-integrated reasoning. We will release our code, models, and data.

Wenjie Li, Yujie Zhang, Haoran Sun, Xingqi He, Hongcheng Gao, Chenglong Ma, Ming Hu, Guankun Wang, Shiyi Yao, Renhao Yang, Hongliang Ren, Lei Wang, Junjun He, Yankai Jiang• 2026

Related benchmarks

TaskDatasetResultRank
Grounded VQAOphVL In-domain (test)
mIoU65.58
13
Grounded VQASurgVidLM Out-of-domain (test)
mIoU34.1
13
Temporal GroundingOphVL In-domain (test)
R@0.381.92
13
Temporal GroundingSurgVidLM Out-of-domain (test)
R@0.355.6
13
Grounded VQAMedVideoCap In-domain (test)
mIoU61.2
13
Perception Visual ReasoningSVU-31K Fine-grained Perception Visual Reasoning 1.0 (test)
BLEU-418.9
11
Temporal Visual ReasoningSVU-31K 1.0 (test)
BLEU-411.3
11
Video descriptionSVU-31K Full Video Description 1.0 (test)
CI4.76
11
Video descriptionSVU-31K 1.0 (test)
CI4
11
Showing 9 of 9 rows

Other info

Follow for update