Unified Concept Editing in Diffusion Models
About
Text-to-image models suffer from various safety issues that may limit their suitability for deployment. Previous methods have separately addressed individual issues of bias, copyright, and offensive content in text-to-image models. However, in the real world, all of these issues appear simultaneously in the same model. We present a method that tackles all issues with a single approach. Our method, Unified Concept Editing (UCE), edits the model without training using a closed-form solution, and scales seamlessly to concurrent edits on text-conditional diffusion models. We demonstrate scalable simultaneous debiasing, style erasure, and content moderation by editing text-to-image projections, and we present extensive experiments demonstrating improved efficacy and scalability over prior work. Our code is available at https://unified.baulab.info
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Text-to-Image Generation | MS-COCO (val) | FID41.55 | 202 | |
| Text-to-Image Generation | MS-COCO | FID18.51 | 131 | |
| Continual Concept Learning | 10 Sequential Concepts (test) | UA100 | 70 | |
| Text-to-Image Generation | MS-COCO (30K) | FID (30K)22.87 | 62 | |
| Text-to-Image Generation | COCO | FID77.41 | 61 | |
| Text-to-Image Generation | MSCOCO 30K | FID77.72 | 54 | |
| Text-to-Image Generation | MS-COCO 30k (val) | FID14.04 | 42 | |
| Concept Erasure | Van Gogh style | FID16.31 | 39 | |
| Nudity Erasure | I2P | Total Count253 | 38 | |
| Artistic Style Erasure | SD Other Class artistic styles 1.4 (test) | Preservation Drop0.7 | 36 |