Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

RESPOND: Risk-Enhanced Structured Pattern for LLM-driven Online Node-level Decision-making

About

Current LLM-based driving agents that rely on unstructured plain-text memory suffer from low-precision scene retrieval and inefficient reflection. To address this limitation, we present RESPOND, a structured decision-making framework for LLM-driven agents grounded in explicit risk patterns. RESPOND represents each ego-centric scene using a unified 5 by 3 matrix that encodes spatial topology and road constraints, enabling consistent and reliable retrieval of spatial risk configurations. Based on this representation, a hybrid rule and LLM decision pipeline is developed with a two-tier memory mechanism. In high-risk contexts, exact pattern matching enables rapid and safe reuse of verified actions, while in low-risk contexts, sub-pattern matching supports personalized driving style adaptation. In addition, a pattern-aware reflection mechanism abstracts tactical corrections from crash and near-miss frames to update structured memory, achieving one-crash-to-generalize learning. Extensive experiments demonstrate the effectiveness of RESPOND. In highway-env, RESPOND outperforms state-of-the-art LLM-based and reinforcement learning based driving agents while producing substantially fewer collisions. With step-wise human feedback, the agent acquires a Sporty driving style within approximately 20 decision steps through sub-pattern abstraction. For real-world validation, RESPOND is evaluated on 53 high-risk cut-in scenarios extracted from the HighD dataset. For each event, intervention is applied immediately before the cut-in and RESPOND re-decides the driving action. Compared to recorded human behavior, RESPOND reduces subsequent risk in 84.9 percent of scenarios, demonstrating its practical feasibility under real-world driving conditions. These results highlight RESPONDs potential for autonomous driving, personalized driving assistance, and proactive hazard mitigation.

Dan Chen, Heye Huang, Tiantian Chen, Zheng Li, Yongji Li, Yuhui Xu, Sikai Chen• 2025

Related benchmarks

TaskDatasetResultRank
Autonomous Drivinghighway-env lane-4-density-2.0
Success Rate1
6
Autonomous Drivinghighway-env Lane-4 Density 2.0 (standard setting)
Success Rate (SR)100
5
Autonomous Drivinghighway-env Lane-5 Density intermediate difficulty 2.5
Success Rate (SR)80
4
Autonomous Drivinghighway-env Lane-5 Density 3.0 (most complex)
Success Rate70
4
Autonomous driving decision-makingHighD high-risk scenarios
Decision Count45
3
Showing 5 of 5 rows

Other info

Follow for update