Beyond Fixed Anchors: Precisely Erasing Concepts with Sibling Exclusive Counterparts
About
Existing concept erasure methods for text-to-image diffusion models commonly rely on fixed anchor strategies, which often lead to critical issues such as concept re-emergence and erosion. To address this, we conduct causal tracing to reveal the inherent sensitivity of erasure to anchor selection and define Sibling Exclusive Concepts as a superior class of anchors. Based on this insight, we propose \textbf{SELECT} (Sibling-Exclusive Evaluation for Contextual Targeting), a dynamic anchor selection framework designed to overcome the limitations of fixed anchors. Our framework introduces a novel two-stage evaluation mechanism that automatically discovers optimal anchors for precise erasure while identifying critical boundary anchors to preserve related concepts. Extensive evaluations demonstrate that SELECT, as a universal anchor solution, not only efficiently adapts to multiple erasure frameworks but also consistently outperforms existing baselines across key performance metrics, averaging only 4 seconds for anchor mining of a single concept.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Art Style Erasure | Artist style and content prompts 5 groups SD v1.4 based (test) | CS Style30.189 | 40 | |
| Art Style Erasure | Artist style and content prompt groups v1.4 (test) | Lgene71.5 | 16 |