Learning to Collide: An Adaptive Safety-Critical Scenarios Generating Method
About
Long-tail and rare event problems become crucial when autonomous driving algorithms are applied in the real world. For the purpose of evaluating systems in challenging settings, we propose a generative framework to create safety-critical scenarios for evaluating specific task algorithms. We first represent the traffic scenarios with a series of autoregressive building blocks and generate diverse scenarios by sampling from the joint distribution of these blocks. We then train the generative model as an agent (or a generator) to investigate the risky distribution parameters for a given driving algorithm being evaluated. We regard the task algorithm as an environment (or a discriminator) that returns a reward to the agent when a risky scenario is generated. Through the experiments conducted on several scenarios in the simulation, we demonstrate that the proposed framework generates safety-critical scenarios more efficiently than grid search or human design methods. Another advantage of this method is its adaptiveness to the routes and parameters.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Autonomous Driving Model Testing | CARLA SafeBench (Multiple Regions) | Compliance Ratio (CR)58.4 | 11 | |
| Safety-critical scenario generation | SafeBench | Collision Rate: Straight Obstacle30 | 8 | |
| Traffic Scenario Simulation | AccidentSim Dual-Vehicle Scenarios | Straight Obstacle Error Rate12 | 7 | |
| Traffic Scenario Simulation | AccidentSim Triple-Vehicle Scenarios | Straight Obstacle Avoidance0.22 | 7 | |
| Traffic Scenario Simulation | AccidentSim Multi-Vehicle Scenarios | Straight Obstacle Rate28 | 7 | |
| Traffic Scenario Simulation | AccidentSim Overall | Average Value30.4 | 7 | |
| Scene Criticality Generation | SafeBench | Collision Rate (CN)58 | 6 | |
| Safety evaluation of autonomous driving | SafeBench critical scenarios | Collision Rate (SO)12 | 5 |