Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

InstantRestore: Single-Step Personalized Face Restoration with Shared-Image Attention

About

Face image restoration aims to enhance degraded facial images while addressing challenges such as diverse degradation types, real-time processing demands, and, most crucially, the preservation of identity-specific features. Existing methods often struggle with slow processing times and suboptimal restoration, especially under severe degradation, failing to accurately reconstruct finer-level identity details. To address these issues, we introduce InstantRestore, a novel framework that leverages a single-step image diffusion model and an attention-sharing mechanism for fast and personalized face restoration. Additionally, InstantRestore incorporates a novel landmark attention loss, aligning key facial landmarks to refine the attention maps, enhancing identity preservation. At inference time, given a degraded input and a small (~4) set of reference images, InstantRestore performs a single forward pass through the network to achieve near real-time performance. Unlike prior approaches that rely on full diffusion processes or per-identity model tuning, InstantRestore offers a scalable solution suitable for large-scale applications. Extensive experiments demonstrate that InstantRestore outperforms existing methods in quality and speed, making it an appealing choice for identity-preserving face restoration.

Howard Zhang, Yuval Alaluf, Sizhuo Ma, Achuta Kadambi, Jian Wang, Kfir Aberman• 2024

Related benchmarks

TaskDatasetResultRank
Face Video RestorationVFHQ heavy degradation (test)
PSNR23.909
11
Face Video RestorationVFHQ (test)
PSNR25.884
11
Video Face RestorationVFHQ milder degradation settings (test)
PSNR27.734
11
Face RestorationFFHQ-Ref Severe
Ref-ArcFace0.473
11
Image Super-resolutionFace
PSNR26.2245
11
Face RestorationSame-age synthetic (test)
PSNR25.6
9
Face RestorationCross-age synthetic (test)
PSNR25.77
9
Face RestorationReal-world face restoration
NIQE5.84
9
Face RestorationCelebHQRef 100
Ref-ArcFace Score51
6
Reference-guided Face RestorationCelebA Ref (test)
GT-AdaFace0.797
6
Showing 10 of 14 rows

Other info

Follow for update