Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

VideoCoF: Unified Video Editing with Temporal Reasoner

About

Existing video editing methods face a critical trade-off: expert models offer precision but rely on task-specific priors like masks, hindering unification; conversely, unified temporal in-context learning models are mask-free but lack explicit spatial cues, leading to weak instruction-to-region mapping and imprecise localization. To resolve this conflict, we propose VideoCoF, a novel Chain-of-Frames approach inspired by Chain-of-Thought reasoning. VideoCoF enforces a ``see, reason, then edit" procedure by compelling the video diffusion model to first predict reasoning tokens (edit-region latents) before generating the target video tokens. This explicit reasoning step removes the need for user-provided masks while achieving precise instruction-to-region alignment and fine-grained video editing. Furthermore, we introduce a RoPE alignment strategy that leverages these reasoning tokens to ensure motion alignment and enable length extrapolation beyond the training duration. We demonstrate that with a minimal data cost of only 50k video pairs, VideoCoF achieves state-of-the-art performance on VideoCoF-Bench, validating the efficiency and effectiveness of our approach. Our code, weight, data are available at https://github.com/knightyxp/VideoCoF.

Xiangpeng Yang, Ji Xie, Yiyuan Yang, Yue Ma, Yan Huang, Min Xu, Qiang Wu• 2025

Related benchmarks

TaskDatasetResultRank
Generation QualityPointBench
Success Rate (%)10
18
Instruction-Guided Video EditingVideoCoF-Bench 1.0 (test)
Instruct Follow Score8.97
6
Object ReplacementOcclusion-Bench
Instance FID (Frame)22.98
6
Object AdditionOcclusion-Bench
Instance FID (Frame)21.46
6
Object RemovalOcclusion-Bench
Instance Fidelity (Frame)22.03
6
Video EditingMiVE Complex
SSIM65.6
5
Video Editingsimple scenario
IA7.53
5
Video EditingMiVE Simple
Gemini Mean Score7.58
5
Video EditingComplex scenario
IA Score7.38
5
Video EditingMiVE Complex
Gemini Score6.3
5
Showing 10 of 11 rows

Other info

GitHub

Follow for update