CoCoSI: Collaborative Cognitive Map Construction for Spatial Intelligence
About
Spatial intelligence is a key frontier for multimodal large language models (MLLMs), enabling them to reason about the physical world from visual experience. Inspired by human spatial cognition, recent approaches construct grid-based cognitive maps from multi-frame visual inputs to maintain coherent spatial representations over time. However, limited context lengths still challenge spatial understanding, while existing methods, such as long-context modeling and external memory, often require architectural changes, memory modules, or finetuning, limiting their applicability to off-the-shelf pretrained MLLMs. This motivates a lightweight, model-agnostic method for preserving spatial information beyond the native context window. To this end, we propose a plug-and-play multi-agent framework that collaboratively constructs cognitive maps as structured spatial memory, enhancing the spatial understanding of arbitrary pretrained MLLMs without architectural modification or additional training. Our framework features local-global agent coordination, cognitive map construction with atomic commits, and cross-agent verification. Extensive experiments demonstrate that our method achieves superior performance on spatial understanding tasks while remaining fully training-free. Code will be released.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Video Spatial Understanding | VSI-Bench 500 QA 1.0 | Accuracy49.7 | 25 | |
| Video Spatial Understanding | MMSI-Bench 500 QA 1.0 | Accuracy41 | 15 | |
| Video Spatial Understanding | MMSI-Bench 500 QA subset | Accuracy35.3 | 5 | |
| Video Spatial Understanding | MMSI-Bench 500 QA pairs | Accuracy41 | 5 |