Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

CRePE: Curved Ray Expectation Positional Encoding for Unified-Camera-Controlled Video Generation

About

Camera-conditioned video generation requires positional encoding that remains reliable under changes in camera motion, lens configuration, and scene structure. However, existing attention-level camera encodings either provide ray-only camera signals or rely on pinhole camera geometry, limiting their applicability to general camera control under the Unified Camera Model, including wide-angle and fisheye lenses. To address this limitation, we propose Curved Ray Expectation Positional Encoding (CRePE). CRePE represents each image token as a depth-aware positional distribution along its source ray, providing a Unified Camera Model-compatible positional encoding that captures the projected-path geometry induced by wide-angle and fisheye cameras. CRePE is implemented through a Geometric Attention Adapter added to frozen video DiTs, injecting token-wise scene-distance information into selected attention layers and stabilizing it with pseudo supervision from a monocular geometry foundation model. This design leads to more stable camera control and improves several geometry-aware and perceptual-quality metrics, while remaining competitive on video-quality metrics. Controlled positional-encoding ablations show a better overall average rank than a RayRoPE-style endpoint PE baseline, demonstrating the effectiveness of UCM-aware projected-path integration across diverse camera models. Furthermore, by extending the same positional-encoding pathway to external geometry control through Radial MixForcing, CRePE supports external radial-map control for scene-geometry-conditioned generation and source-video motion transfer beyond camera control.

Seonghyun Jin, Youngmin Kim, Sunwoo Park, Jong Chul Ye• 2026

Related benchmarks

TaskDatasetResultRank
Camera-conditioned video generationPanShot merged pinhole and non-pinhole (test)
CS-I98.772
3
Camera-controlled Video GenerationPanShot merged pinhole and non-pinhole
Camera Motion Consistency18.15
3
Video GenerationVBench
Aesthetic Score0.5634
3
Camera-controlled Video GenerationPanShot Non-pinhole
CamMC19.02
3
Camera-controlled Video GenerationPanShot (Pinhole)
CamMC15.33
3
Showing 5 of 5 rows

Other info

Follow for update