Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Unified Concept Editing in Diffusion Models

About

Text-to-image models suffer from various safety issues that may limit their suitability for deployment. Previous methods have separately addressed individual issues of bias, copyright, and offensive content in text-to-image models. However, in the real world, all of these issues appear simultaneously in the same model. We present a method that tackles all issues with a single approach. Our method, Unified Concept Editing (UCE), edits the model without training using a closed-form solution, and scales seamlessly to concurrent edits on text-conditional diffusion models. We demonstrate scalable simultaneous debiasing, style erasure, and content moderation by editing text-to-image projections, and we present extensive experiments demonstrating improved efficacy and scalability over prior work. Our code is available at https://unified.baulab.info

Rohit Gandikota, Hadas Orgad, Yonatan Belinkov, Joanna Materzy\'nska, David Bau• 2023

Related benchmarks

TaskDatasetResultRank
Text-to-Image GenerationMS-COCO (val)
FID41.55
202
Text-to-Image GenerationMS-COCO
FID18.51
131
Continual Concept Learning10 Sequential Concepts (test)
UA100
70
Text-to-Image GenerationMS-COCO (30K)
FID (30K)22.87
62
Text-to-Image GenerationCOCO
FID77.41
61
Text-to-Image GenerationMSCOCO 30K
FID77.72
54
Text-to-Image GenerationMS-COCO 30k (val)
FID14.04
42
Concept ErasureVan Gogh style
FID16.31
39
Nudity ErasureI2P
Total Count253
38
Artistic Style ErasureSD Other Class artistic styles 1.4 (test)
Preservation Drop0.7
36
Showing 10 of 223 rows
...

Other info

Follow for update