Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

InstantRestore: Single-Step Personalized Face Restoration with Shared-Image Attention

About

Face image restoration aims to enhance degraded facial images while addressing challenges such as diverse degradation types, real-time processing demands, and, most crucially, the preservation of identity-specific features. Existing methods often struggle with slow processing times and suboptimal restoration, especially under severe degradation, failing to accurately reconstruct finer-level identity details. To address these issues, we introduce InstantRestore, a novel framework that leverages a single-step image diffusion model and an attention-sharing mechanism for fast and personalized face restoration. Additionally, InstantRestore incorporates a novel landmark attention loss, aligning key facial landmarks to refine the attention maps, enhancing identity preservation. At inference time, given a degraded input and a small (~4) set of reference images, InstantRestore performs a single forward pass through the network to achieve near real-time performance. Unlike prior approaches that rely on full diffusion processes or per-identity model tuning, InstantRestore offers a scalable solution suitable for large-scale applications. Extensive experiments demonstrate that InstantRestore outperforms existing methods in quality and speed, making it an appealing choice for identity-preserving face restoration.

Howard Zhang, Yuval Alaluf, Sizhuo Ma, Achuta Kadambi, Jian Wang, Kfir Aberman• 2024

Related benchmarks

TaskDatasetResultRank
Image Super-resolutionFace
PSNR26.2245
11
Face RestorationSame-age synthetic (test)
PSNR25.6
9
Face RestorationCross-age synthetic (test)
PSNR25.77
9
Face RestorationReal-world face restoration
NIQE5.84
9
Showing 4 of 4 rows

Other info

Follow for update