Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Panacea+: Panoramic and Controllable Video Generation for Autonomous Driving

About

The field of autonomous driving increasingly demands high-quality annotated video training data. In this paper, we propose Panacea+, a powerful and universally applicable framework for generating video data in driving scenes. Built upon the foundation of our previous work, Panacea, Panacea+ adopts a multi-view appearance noise prior mechanism and a super-resolution module for enhanced consistency and increased resolution. Extensive experiments show that the generated video samples from Panacea+ greatly benefit a wide range of tasks on different datasets, including 3D object tracking, 3D object detection, and lane detection tasks on the nuScenes and Argoverse 2 dataset. These results strongly prove Panacea+ to be a valuable data generation framework for autonomous driving.

Yuqing Wen, Yucheng Zhao, Yingfei Liu, Binyuan Huang, Fan Jia, Yanhui Wang, Chi Zhang, Tiancai Wang, Xiaoyan Sun, Xiangyu Zhang• 2024

Related benchmarks

TaskDatasetResultRank
3D Object DetectionnuScenes (val)
NDS49.2
981
3D Object DetectionnuScenes (val)
NDS27.73
249
Video GenerationnuScenes (val)
FVD139
101
3D Object DetectionnuScenes
mAP (All)13.72
41
Driving Scene GenerationnuScenes (val)
FID15.5
27
PlanningnuScenes
L2 Error (1s)0.58
26
Map SegmentationnuScenes
Drivable Area52.37
19
PlanningnuScenes (val)
L2 Error (1s)0.58
16
BeV SegmentationnuScenes (val)--
16
Video GenerationnuScenes v1.0 (val)
FVD139
13
Showing 10 of 19 rows

Other info

Follow for update