Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

DiffusionFake: Enhancing Generalization in Deepfake Detection via Guided Stable Diffusion

About

The rapid progress of Deepfake technology has made face swapping highly realistic, raising concerns about the malicious use of fabricated facial content. Existing methods often struggle to generalize to unseen domains due to the diverse nature of facial manipulations. In this paper, we revisit the generation process and identify a universal principle: Deepfake images inherently contain information from both source and target identities, while genuine faces maintain a consistent identity. Building upon this insight, we introduce DiffusionFake, a novel plug-and-play framework that reverses the generative process of face forgeries to enhance the generalization of detection models. DiffusionFake achieves this by injecting the features extracted by the detection model into a frozen pre-trained Stable Diffusion model, compelling it to reconstruct the corresponding target and source images. This guided reconstruction process constrains the detection network to capture the source and target related features to facilitate the reconstruction, thereby learning rich and disentangled representations that are more resilient to unseen forgeries. Extensive experiments demonstrate that DiffusionFake significantly improves cross-domain generalization of various detector architectures without introducing additional parameters during inference. Our Codes are available in https://github.com/skJack/DiffusionFake.git.

Ke Sun, Shen Chen, Taiping Yao, Hong Liu, Xiaoshuai Sun, Shouhong Ding, Rongrong Ji• 2024

Related benchmarks

TaskDatasetResultRank
Deepfake DetectionCDFv1, CDFv2, DFD, DFDCP, DFDC (test)
Overall Average Score84
74
Face Forgery DetectionDFDCP
Frame-level AUC81
64
Frame-level Deepfake DetectionDFD
AUC91.71
42
Frame-level Deepfake DetectionDFDC-P
AUC80.95
28
Frame-level Face Forgery DetectionWild Deepfake
AUC80.14
24
Face Forgery DetectionCDF v2 (test)
Frame-Level AUC80.5
23
Face Forgery DetectionDFD (test)
Frame-Level AUC90.4
21
Deepfake DetectionCDF v2 (test)
Video-level AUC0.8317
19
Frame-level Deepfake DetectionCeleb-DF
AUC0.8317
18
Frame-level Deepfake DetectionDiffSwap
AUC (%)86.98
13
Showing 10 of 17 rows

Other info

Code

Follow for update