Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Toward Trustworthy Portrait Editing: Evaluation of Demographic Misrepresentation in I2I Models

About

Instruction-guided image-to-image (I2I) editors are increasingly used in consumer and professional visual workflows, where trustworthiness depends not only on prompt compliance but also on equitable preservation of identity-relevant attributes. We formalize two failure modes: Soft Erasure, where requested edits are weakly realized or silently suppressed, and Stereotype Replacement, where edits introduce unrequested, stereotype-consistent demographic attributes. Using a controlled benchmark of 5,040 edited portraits, we evaluate these failures across three recent open-weight editors with vision-language model scoring and human evaluation. Our results show that identity-preservation failures are pervasive and demographically uneven. In particular, 62--71% of outputs exhibit skin lightening, with Indian and Black source portraits affected at 72--75%, compared with 44% for White source portraits, indicating output-level drift toward lighter or more White-presenting appearances when identity constraints are underspecified. In a mitigation case study, prompt-level appearance constraints reduce race-change scores for non-White source portraits by up to 1.48 points, while leaving White source portraits largely unchanged, without modifying model weights. These findings show that identity preservation is not a uniform property of I2I portrait editing systems, but an unevenly distributed trustworthiness failure with direct social consequences. At deployment scale, such silent distortions can shape AI-mediated self-representation and reinforce representational disparities. We introduce a controlled audit protocol for fairness-aware evaluation and governance of generative editing systems. Project page: https://seochan99.github.io/i2i-demographic-bias

Huichan Seo, Minki Hong, Sieun Choi, Jihie Kim, Jean Oh• 2026

Related benchmarks

TaskDatasetResultRank
Text-to-Image DebiasingWinobias--
6
Image-to-Image EditingWinoBias adapted for I2I editing (test)--
3
Instruction-based Image EditingFairFace--
3
Showing 3 of 3 rows

Other info

Follow for update