Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

Counterfactual Data Augmentation for Mitigating Gender Stereotypes in Languages with Rich Morphology

About

Gender stereotypes are manifest in most of the world's languages and are consequently propagated or amplified by NLP systems. Although research has focused on mitigating gender stereotypes in English, the approaches that are commonly employed produce ungrammatical sentences in morphologically rich languages. We present a novel approach for converting between masculine-inflected and feminine-inflected sentences in such languages. For Spanish and Hebrew, our approach achieves F1 scores of 82% and 73% at the level of tags and accuracies of 90% and 87% at the level of forms. By evaluating our approach using four different languages, we show that, on average, it reduces gender stereotyping by a factor of 2.5 without any sacrifice to grammaticality.

Ran Zmigrod, Sabrina J. Mielke, Hanna Wallach, Ryan Cotterell• 2019

Related benchmarks

TaskDatasetResultRank
Counterfactual Input EvaluationCrowS-Pairs
SS55.35
33
Stereotype Bias EvaluationStereoSet Gender
LMS Score85.47
15
Gender bias evaluationSEAT
SEAT 60.596
13
Stereotypical Bias EvaluationStereoSet (dev)
Overall LMS Score83.466
12
Bias EvaluationCrow-S
Score56.107
9
Showing 5 of 5 rows

Other info

Follow for update