Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

DiG: Differential Grounding for Enhancing Fine-Grained Perception in Multimodal Large Language Model

About

Multimodal Large Language Models have achieved impressive performance on a variety of vision-language tasks, yet their fine-grained visual perception and precise spatial reasoning remain limited. In this work, we introduce DiG (Differential Grounding), a novel proxy task framework where MLLMs learn fine-grained perception by identifying and localizing all differences between similar image pairs without prior knowledge of their number. To support scalable training, we develop an automated 3D rendering-based data generation pipeline that produces high-quality paired images with fully controllable discrepancies. To address the sparsity of difference signals, we further employ curriculum learning that progressively increases complexity from single to multiple differences, enabling stable optimization. Extensive experiments demonstrate that DiG significantly improves model performance across a variety of visual perception benchmarks and that the learned fine-grained perception skills transfer effectively to standard downstream tasks, including RefCOCO, RefCOCO+, RefCOCOg, and general multimodal perception benchmarks. Our results highlight differential grounding as a scalable and robust approach for advancing fine-grained visual reasoning in MLLMs.

Zhou Tao, Shida Wang, Yongxiang Hua, Haoyu Cao, Linli Xu• 2025

Related benchmarks

TaskDatasetResultRank
Visual Question AnsweringTextVQA
Accuracy77.5
1117
Object Hallucination EvaluationPOPE
Accuracy88.5
935
Multimodal UnderstandingMM-Vet
MM-Vet Score77.3
418
Multimodal UnderstandingMMBench
Accuracy87.2
367
Multimodal UnderstandingMMStar--
197
Diagram UnderstandingAI2D
Accuracy84.1
167
Multimodal UnderstandingMME--
158
Spatial ReasoningVisual Spatial Reasoning (VSR)
Accuracy83.9
48
Multimodal Visual PerceptionMMVP
Accuracy79
44
Multimodal PerceptionVSTAR
Accuracy81.2
18
Showing 10 of 17 rows

Other info

Follow for update