CogDriver: Integrating Cognitive Inertia for Temporally Coherent Planning in Autonomous Driving
About
The pursuit of autonomous agents capable of temporally coherent planning is hindered by a fundamental flaw in current vision-language models (VLMs): they lack cognitive inertia. Operating on isolated snapshots, these models cannot form a continuous understanding of the environment, leading to erratic decision jitter and a failure to execute complex, multi-step maneuvers. To remedy this, we introduce CogDriver, a framework designed to build a stable internal representation by instilling this crucial cognitive property. Our work makes two key contributions: (1) We present CogDriver-Data, a large-scale vision-language-action dataset whose narrative annotations provide the supervisory signal for learning temporal dynamics and persistent intent. (2) We develop the CogDriver-Agent, an architecture featuring a sparse temporal memory to maintain a stable internal state. This is enabled by a spatiotemporal knowledge distillation approach that explicitly teaches decision coherence. Comprehensive experiments validate our paradigm: CogDriver-Agent achieves a 22% increase in the closed-loop Driving Score on Bench2Drive and a 21% reduction in mean L2 error on nuScenes, establishing a new state-of-the-art. These significant gains in both long-term decision-making and imitation accuracy provide strong evidence that our agent successfully maintains a temporally coherent internal state, bridging the gap toward more reliable autonomous driving. Project link: https://ocean-luna.github.io/CogDriver.github.io/.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Closed-loop Autonomous Driving | Bench2Drive | Driving Score (DS)78.21 | 74 | |
| Open-loop planning | NuScenes v1.0 (test) | L2 Error (1s)0.15 | 50 | |
| Open-loop planning | Bench2Drive | Average L2 Error0.63 | 47 | |
| Visual Question Answering | CogDriver-nuScenes | CIDEr92.39 | 9 | |
| Visual Question Answering | CogDriver-Bench2Drive | CIDEr95.46 | 9 | |
| Visual Question Answering | OmniDrive (test) | CI-r86.67 | 6 |