LVTINO: LAtent Video consisTency INverse sOlver for High Definition Video Restoration
About
Computational imaging methods increasingly rely on powerful generative diffusion models to tackle challenging image restoration tasks. In particular, state-of-the-art zero-shot image inverse solvers leverage distilled text-to-image latent diffusion models (LDMs) to achieve unprecedented accuracy and perceptual quality with high computational efficiency. However, extending these advances to high-definition video restoration remains a significant challenge, due to the need to recover fine spatial detail while capturing subtle temporal dependencies. Consequently, methods that naively apply image-based LDM priors on a frame-by-frame basis often result in temporally inconsistent reconstructions. We address this challenge by leveraging recent advances in Video Consistency Models (VCMs), which distill video latent diffusion models into fast generators that explicitly capture temporal causality. Building on this foundation, we propose LVTINO, the first zero-shot or plug-and-play inverse solver for high definition video restoration with priors encoded by VCMs. Our conditioning mechanism bypasses the need for automatic differentiation and achieves state-of-the-art video reconstruction quality with only a few neural function evaluations, while ensuring strong measurement consistency and smooth temporal transitions across frames. Extensive experiments on a diverse set of video inverse problems show significant perceptual improvements over current state-of-the-art methods that apply image LDMs frame by frame, establishing a new benchmark in both reconstruction fidelity and computational efficiency. The code is available on GitHub.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Video Restoration (Problem B: Temporal blur + Spatial SRx8) | Adobe240 | FVMD42.65 | 5 | |
| Video Restoration Efficiency | 25-frame video clip 1280 x 768 resolution | Time (s)105 | 5 | |
| Video Restoration (Problem C: Temp. SRx8 + SRx8) | GoPro 240 | FVMD232.6 | 4 | |
| Video Restoration (Problem C: Temporal SRx8 + Spatial SRx8) | Adobe240 | FVMD602.5 | 4 | |
| Video Restoration (Problem A: Temp. SRx4 + SRx4) | GoPro240 | FVMD189.4 | 3 | |
| Video Restoration (Problem A: Temporal SRx4 + Spatial SRx4) | Adobe240 | FVMD371.1 | 3 | |
| Video Restoration (Problem B: Temp. blur + SRx8) | GoPro240 | FVMD46.2 | 3 | |
| Temporal and Spatial Super-Resolution (x8) | Adobe240 noise sigma=0.01 | FVMD310.6 | 3 | |
| Temporal and Spatial Super-Resolution (x4) | Adobe240 noise sigma=0.01 | FVMD256.5 | 2 |