Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

CoCoSI: Collaborative Cognitive Map Construction for Spatial Intelligence

About

Spatial intelligence is a key frontier for multimodal large language models (MLLMs), enabling them to reason about the physical world from visual experience. Inspired by human spatial cognition, recent approaches construct grid-based cognitive maps from multi-frame visual inputs to maintain coherent spatial representations over time. However, limited context lengths still challenge spatial understanding, while existing methods, such as long-context modeling and external memory, often require architectural changes, memory modules, or finetuning, limiting their applicability to off-the-shelf pretrained MLLMs. This motivates a lightweight, model-agnostic method for preserving spatial information beyond the native context window. To this end, we propose a plug-and-play multi-agent framework that collaboratively constructs cognitive maps as structured spatial memory, enhancing the spatial understanding of arbitrary pretrained MLLMs without architectural modification or additional training. Our framework features local-global agent coordination, cognitive map construction with atomic commits, and cross-agent verification. Extensive experiments demonstrate that our method achieves superior performance on spatial understanding tasks while remaining fully training-free. Code will be released.

Yiming Zhang, Ruoxuan Cao, Zhihang Zhong• 2026

Related benchmarks

TaskDatasetResultRank
Video Spatial UnderstandingVSI-Bench 500 QA 1.0
Accuracy49.7
25
Video Spatial UnderstandingMMSI-Bench 500 QA 1.0
Accuracy41
15
Video Spatial UnderstandingMMSI-Bench 500 QA subset
Accuracy35.3
5
Video Spatial UnderstandingMMSI-Bench 500 QA pairs
Accuracy41
5
Showing 4 of 4 rows

Other info

Follow for update