Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Unified Safe In-context Image Generation in Multimodal Diffusion Transformers via Restricting Unsafe Information Flows

About

Diffusion transformers (DiTs) equipped with multimodal attention (MM-Attn) have become a dominant paradigm for image generation. However, preventing the generation of harmful content remains a critical challenge, particularly in image-to-image (I2I) editing tasks. Existing safety mechanisms are primarily designed for text-to-image (T2I) synthesis or U-Net-based architectures, which limits their effectiveness for unified safety mitigation in DiT-based frameworks. To bridge this gap, we propose Unified Visual Safety Regulator (UVR), a training-free safe generation framework that regulates unsafe semantics in generated images. UVR is grounded in an analysis of attention dynamics from the perspective of information flow in MM-Attn. We identify a task-independent start-up stage, during which unsafe semantics in output patches rapidly emerge and can be accurately localized, followed by task-specific semantic amplification and interference stages, where harmful signals are further propagated and entangled with benign content. Based on these observations, UVR mitigates unsafe generation through unified, targeted attention modulation and explicit restriction of harmful information flow over the identified unsafe output patches. Experiments across various concepts show that UVR achieves state-of-the-art safety performance by achieving 91% and 77% erase rate in image synthesis and editing tasks, while preserving visual quality and fidelity with minimal degradation. Code is available at https://github.com/deng12yx/UVR.

Xiang Yang, Feifei Li, Mi Zhang, Geng Hong, Xiaoyu You, Mi Wen, Min Yang• 2026

Related benchmarks

TaskDatasetResultRank
Image-to-Image EditingImagenHub Instruction-image pairs
VQA Accuracy88.42
14
Safe Text-to-Image GenerationRing-a-Bell--
13
Image-to-Image EditingI2P Sexual
Total Unsafe Images46
7
Text-to-Image GenerationI2P sexual prompts
Total Unsafe Count40
7
Text-to-Image GenerationUnsafe-1K adversarial prompts
Total Unsafe Images (T)97
7
Text-to-Image GenerationMS-COCO 2014
VQA Score87.44
7
Safe Text-to-Image SynthesisFLUX.1 12B (dev)
Inference Time (s)25
7
Image-to-Image EditingUnsafe-1K (adversarial)
Total Unsafe Images Count151
7
Safe Text-to-Image SynthesisNude Concept FLUX.1-schnell
CLIP Similarity31.51
2
Text-to-Image Synthesis SafetyP4D (Prompt4Debugging)
Total Violation Count16
2
Showing 10 of 11 rows

Other info

Follow for update