Panacea+: Panoramic and Controllable Video Generation for Autonomous Driving
About
The field of autonomous driving increasingly demands high-quality annotated video training data. In this paper, we propose Panacea+, a powerful and universally applicable framework for generating video data in driving scenes. Built upon the foundation of our previous work, Panacea, Panacea+ adopts a multi-view appearance noise prior mechanism and a super-resolution module for enhanced consistency and increased resolution. Extensive experiments show that the generated video samples from Panacea+ greatly benefit a wide range of tasks on different datasets, including 3D object tracking, 3D object detection, and lane detection tasks on the nuScenes and Argoverse 2 dataset. These results strongly prove Panacea+ to be a valuable data generation framework for autonomous driving.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| 3D Object Detection | nuScenes (val) | NDS49.2 | 981 | |
| 3D Object Detection | nuScenes (val) | NDS27.73 | 249 | |
| Video Generation | nuScenes (val) | FVD139 | 101 | |
| 3D Object Detection | nuScenes | mAP (All)13.72 | 41 | |
| Driving Scene Generation | nuScenes (val) | FID15.5 | 27 | |
| Planning | nuScenes | L2 Error (1s)0.58 | 26 | |
| Map Segmentation | nuScenes | Drivable Area52.37 | 19 | |
| Planning | nuScenes (val) | L2 Error (1s)0.58 | 16 | |
| BeV Segmentation | nuScenes (val) | -- | 16 | |
| Video Generation | nuScenes v1.0 (val) | FVD139 | 13 |