Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Beyond Surface Artifacts: Capturing Shared Latent Forgery Knowledge Across Modalities

About

As generative artificial intelligence evolves, deepfake attacks have escalated from single-modality manipulations to complex, multimodal threats. Existing forensic techniques face a severe generalization bottleneck: by relying excessively on superficial, modality-specific artifacts, they neglect the shared latent forgery knowledge hidden beneath variable physical appearances. Consequently, these models suffer catastrophic performance degradation when confronted with unseen "dark modalities." To break this limitation, this paper introduces a paradigm shift that redefines multimodal forensics from conventional "feature fusion" to "modality generalization." We propose the first modality-agnostic forgery (MAF) detection framework. By explicitly decoupling modality-specific styles, MAF precisely extracts the essential, cross-modal latent forgery knowledge. Furthermore, we define two progressive dimensions to quantify model generalization: transferability toward semantically correlated modalities (Weak MAF), and robustness against completely isolated signals of "dark modality" (Strong MAF). To rigorously assess these generalization limits, we introduce the DeepModal-Bench benchmark, which integrates diverse multimodal forgery detection algorithms and adapts state-of-the-art generalized learning methods. This study not only empirically proves the existence of universal forgery traces but also achieves significant performance breakthroughs on unknown modalities via the MAF framework, offering a pioneering technical pathway for universal multimodal defense.

Jingtong Dou, Chuancheng Shi, Jian Wang, Fei Shen, Zhiyong Wang, Tat-Seng Chua• 2026

Related benchmarks

TaskDatasetResultRank
ClassificationLAV-DF leave-one-modality-out cross-validation--
39
ClassificationFakeavcele leave-one-modality-out (cross-val)--
39
ClassificationCele+Asv leave-one-modality-out (val)--
39
Multimodal Forgery DetectionLAV-DF (test)--
15
Multimodal Forgery DetectionFakeavcele (test)--
3
Multimodal Forgery DetectionCele+Asv (test)--
2
Showing 6 of 6 rows

Other info

Follow for update