Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

AutoWeather4D: Autonomous Driving Video Weather Conversion via G-Buffer Dual-Pass Editing

About

Generative video models have significantly advanced the photorealistic synthesis of adverse weather for autonomous driving; however, they consistently demand massive datasets to learn rare weather scenarios. While 3D-aware editing methods alleviate these data constraints by augmenting existing video footage, they are fundamentally bottlenecked by costly per-scene optimization and suffer from inherent geometric and illumination entanglement. In this work, we introduce AutoWeather4D, a feed-forward 3D-aware weather editing framework designed to explicitly decouple geometry and illumination. At the core of our approach is a G-buffer Dual-pass Editing mechanism. The Geometry Pass leverages explicit structural foundations to enable surface-anchored physical interactions, while the Light Pass analytically resolves light transport, accumulating the contributions of local illuminants into the global illumination to enable dynamic 3D local relighting. Extensive experiments demonstrate that AutoWeather4D achieves comparable photorealism and structural consistency to generative baselines while enabling fine-grained parametric physical control, serving as a practical data engine for autonomous driving.

Tianyu Liu, Weitao Xiong, Kunming Luo, Manyuan Zhang, Peng Li, Yuan Liu, Ping Tan• 2026

Related benchmarks

TaskDatasetResultRank
Semantic segmentationDark Zurich
mIoU24.09
41
Semantic segmentationACDC--
17
Weather and Time-of-day ConversionWaymo (test)
CLIP Score0.2586
5
Depth Alignment EvaluationAutoWeather4D 480 generated videos
Depth si-RMSE0.247
5
Video EditingFour weather-specific datasets (480 generated videos) (test)
FVD Score886.8
5
Edge Alignment480 generated videos (test)
Edge F112.9
5
Showing 6 of 6 rows

Other info

GitHub

Follow for update