Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

UCM: Unified Modeling of Camera Control and Memory with Time-aware Positional Encoding Warping for World Models

About

World models based on video generation demonstrate remarkable potential for simulating interactive environments yet suffer from persistent difficulties in two key areas: maintaining long-term content consistency when scenes are revisited and enabling precise camera control from user-specified inputs. Existing methods based on explicit 3D reconstruction often compromise flexibility in unbounded scenarios and struggle to preserve fine-grained structures. Alternative methods rely directly on previously generated frames without establishing explicit spatial correspondence, thereby limiting controllability and consistency. To address these limitations, we present UCM, a novel framework for unified modeling of long-term memory and precise camera control via a time-aware positional encoding warping mechanism. To reduce computational overhead, we design an efficient dual-stream diffusion transformer for high-fidelity generation. Moreover, we introduce a scalable data curation strategy that utilizes point-cloud-based rendering to simulate scene revisiting, enabling training on over 500K monocular videos. Extensive experiments on real-world and synthetic benchmarks demonstrate that UCM significantly outperforms state-of-the-art methods on long-term scene consistency, while achieving precise camera controllability in high-fidelity video generation.

Tianxing Xu, Zixuan Wang, Guangyuan Wang, Li Hu, Zhongyi Zhang, Peng Zhang, Bang Zhang, Songhai Zhang• 2026

Related benchmarks

TaskDatasetResultRank
Video GenerationRealEstate10K Long-duration (test)
RotErr (°)2.32
8
Camera-Controllable Video GenerationRealestate10K, Tanks-and-Temples, and Context-as-Memory 5% (test)
Rotation Error1.01
5
Inference Speed Evaluationlong-term video generation evaluation 561-frame
Gen. Time (s/frame)2.4
4
Showing 3 of 3 rows

Other info

Follow for update