Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

Omni-Weather: Unified Multimodal Foundation Model for Weather Generation and Understanding

About

Weather modeling requires both accurate prediction and mechanistic interpretation, yet existing methods treat these goals in isolation, separating generation from understanding. To address this gap, we present Omni-Weather, the first multimodal foundation model that unifies weather generation and understanding within a single architecture. Omni-Weather integrates a radar encoder for weather generation tasks, followed by unified processing using a shared self-attention mechanism. Moreover, we construct a Chain-of-Thought dataset for causal reasoning in weather generation, enabling interpretable outputs and improved perceptual quality. Extensive experiments show Omni-Weather achieves state-of-the-art performance in both weather generation and understanding. Our findings further indicate that generative and understanding tasks in the weather domain can mutually enhance each other. Omni-Weather also demonstrates the feasibility and value of unifying weather generation and understanding.

Zhiwang Zhou, Yuandong Pu, Xuming He, Yidi Liu, Yixin Chen, Junchao Gong, Xiang Zhuang, Wanghan Xu, Qinglong Cao, Shixiang Tang, Yihao Liu, Wenlong Zhang, Lei Bai• 2025

Related benchmarks

TaskDatasetResultRank
Radar NowcastingSEVIR
SSIM75.1
31
Radar Image UnderstandingRadarQA
Overall Score64.3
5
Radar Sequence UnderstandingRadarQA
Overall61.79
5
Radar InversionSEVIR
R.S2.42
5
Showing 4 of 4 rows

Other info

Follow for update