Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

PixelEyes: Decoupling Perception and Reasoning for Pinpoint Visual Evidence Seeking

About

This paper explores multi-turn visual reasoning and observes that MLLMs repeatedly fail to localize the target, leading to long, redundant trajectories. We attribute this failure to the entanglement of reasoning and perception within a single model, the MLLM reasons and localizes simultaneously, and inaccurate localization triggers additional reasoning turns that bloat the trajectory. To solve this problem, we propose PixelEyes, a multi-turn visual reasoning agent that explicitly decouples reasoning from perception, i.e., the reasoner decides what to look for, while a specialized perception tool answers where it is. Specifically, PixelEyes introduces 1) Mask-guided Visual Search. A referring segmentation model is invoked to provide mask-precise localization, freeing the reasoner from the need to compensate for imprecise grounding. 2) Semantic-region Breadth-first Search (BFS). To eliminate redundant loops caused by repeatedly cropping incorrect sub-regions, we organize exploration as a breadth-first search over semantic regions. To internalize these capabilities, we construct the PixelEyes-6K dataset by resynthesizing expert trajectories from existing data. This explicitly embeds our mask-guided search and BFS logic into the model. We further introduce Pinpoint-Bench, a zero-hint visual search benchmark, i.e., no location cues are provided in the question, with instance-level masks and bounding boxes that separate localization failures from reasoning failures, enabling fine-grained analysis of failure modes such as inattentional blindness. Recent state-of-the-art MLLMs and visual reasoning agents leave large headroom on Pinpoint-Bench, demonstrating its quality and difficulty. Code and models are open-sourced.

Dengxian Gong, Yuanzheng Wu, Haobo Yuan, Zhengdong Hu, Tao Zhang, Yikang Zhou, Shihao Chen, Quanzhu Niu, Kai Wang, Jason Li, Haochen Wang, Lu Qi, Shunping Ji, Ming-Hsuan Yang• 2026

Related benchmarks

TaskDatasetResultRank
General Visual ReasoningMME-RealWorld-Lite
Accuracy59.25
47
Active PerceptionV*
Accuracy94.24
11
Active PerceptionHR-Bench-4K
Accuracy85
11
Active PerceptionHR-Bench-8K
Accuracy83.15
11
Complex visual searchVisualProbe
Success Rate (Hard)59.44
10
Needle-in-a-haystack localizationPinpoint-Bench
Accuracy55.2
10
General and structural reasoningTree Bench
Accuracy48.4
10
Visual SearchPinpoint-Bench
Accuracy54.73
4
Showing 8 of 8 rows

Other info

GitHub

Follow for update