Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

CineTrans: Learning to Generate Videos with Cinematic Transitions via Masked Diffusion Models

About

Despite significant advances in video synthesis, research into multi-shot video generation remains in its infancy. Even with scaled-up models and massive datasets, the shot transition capabilities remain rudimentary and unstable, largely confining generated videos to single-shot sequences. In this work, we introduce CineTrans, a novel framework for generating coherent multi-shot videos with cinematic, film-style transitions. To facilitate insights into the film editing style, we construct a multi-shot video-text dataset Cine250K with detailed shot annotations. Furthermore, our analysis of existing video diffusion models uncovers a correspondence between attention maps in the diffusion model and shot boundaries, which we leverage to design a mask-based control mechanism that enables transitions at arbitrary positions and transfers effectively in a training-free setting. After fine-tuning on our dataset with the mask mechanism, CineTrans produces cinematic multi-shot sequences while adhering to the film editing style, avoiding unstable transitions or naive concatenations. Finally, we propose specialized evaluation metrics for transition control, temporal consistency and overall quality, and demonstrate through extensive experiments that CineTrans significantly outperforms existing baselines across all criteria.

Xiaoxue Wu, Bingjie Gao, Yu Qiao, Yaohui Wang, Xinyuan Chen• 2025

Related benchmarks

TaskDatasetResultRank
Multi-shot Video Generation90 prompts evaluation suite
Type Accuracy39.44
9
Multi-shot Cinematic Video GenerationMulti-shot Cinematic Video Generation (test)
AQ (Aesthetic Quality)56.52
9
Multi-shot Video GenerationGemini 100 multi-shot video prompts 2.5 Pro
Intra-shot Consistency (Subject)0.776
8
Long Video GenerationUser Study Evaluation Set (test)
Visual Consistency6.21
8
Multi-shot Video Generation100 hierarchical prompts with transitions
Inter-shot Semantic Consistency Score80.95
7
Multi-shot Video Generation20 evaluation prompts 1.0 (User Study)
Temporal Consistency Score4.15
7
Multi-shot Cinematic Video GenerationHuman evaluation
VQE13.2
6
Showing 7 of 7 rows

Other info

Follow for update