RAVEN: Long-Horizon Reasoning & Navigation with a Visuo-Spatio-Temporal Memory
About
Long-term robot deployment requires a compact and scalable memory that preserves fine-grained visual semantics, grounds observations in space and time, and enables efficient storage and retrieval. In this paper, we propose RAVEN, an agentic memory system for long-horizon robotic question answering and navigation. RAVEN stores visual embeddings with pose and time in a vector database, and grounds retrieval in a spatial map to answer queries and navigate to goals. By operating directly on visual embeddings, RAVEN avoids lossy image-to-text captioning and enables accurate semantic, spatial, and temporal retrieval at scale. Across several simulated and real-world video question-answering benchmarks, RAVEN consistently surpasses caption-based memory systems and matches frontier VLMs on long-horizon tasks at 10$\times$ lower retrieval cost. Finally, we instantiate RAVEN on a Unitree Go1 robot for the task of long-horizon navigation for natural language goal-reaching, and show successful deployment over several large indoor environments.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Video Question Answering | RAVEN-QA Simple Queries | Accuracy100 | 64 | |
| Video Question Answering | RAVEN-QA Hard Queries | Accuracy95.1 | 64 | |
| Navigation Question Answering | RAVEN-QA Real-World Robot Tasks | Overall Accuracy97.14 | 28 | |
| Navigation Question Answering | RAVEN-QA Habitat Simulation Tasks | Overall Accuracy86.84 | 28 | |
| Navigation Visual Question Answering | NaVQA Short (S) horizon | Descriptive Accuracy76.2 | 23 | |
| Navigation Visual Question Answering | NaVQA Medium (M) horizon | Descriptive Accuracy60.5 | 7 | |
| Memory-based navigation | FindingDory Long-Horizon (ep_91 to ep_100) | Overall Accuracy32.2 | 6 | |
| Memory-based navigation | FindingDory Full Dataset (ep_1 to ep_100) | Overall Accuracy35.1 | 6 | |
| Navigation Visual Question Answering | NaVQA Long (L) horizon | Descriptive Accuracy80 | 2 |