Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

K/DA: Automated Data Generation Pipeline for Detoxifying Implicitly Offensive Language in Korean

About

Language detoxification involves removing toxicity from offensive language. While a neutral-toxic paired dataset provides a straightforward approach for training detoxification models, creating such datasets presents several challenges: i) the need for human annotation to build paired data, and ii) the rapid evolution of offensive terms, rendering static datasets quickly outdated. To tackle these challenges, we introduce an automated paired data generation pipeline, called K/DA. This pipeline is designed to generate offensive language with implicit offensiveness and trend-aligned slang, making the resulting dataset suitable for detoxification model training. We demonstrate that the dataset generated by K/DA exhibits high pair consistency and greater implicit offensiveness compared to existing Korean datasets, and also demonstrates applicability to other languages. Furthermore, it enables effective training of a high-performing detoxification model with simple instruction fine-tuning.

Minkyeong Jeon, Hyemin Jeong, Yerang Kim, Jiyoung Kim, Jae Hyeon Cho, Byung-Jun Lee• 2025

Related benchmarks

TaskDatasetResultRank
Language DetoxificationOurs (test)
Overall Offensiveness Score1.145
5
Language DetoxificationKOLD (test)
Overall Offensiveness Score1.606
5
Language DetoxificationBEEP (test)
Overall Offensiveness1.58
5
Human EvaluationK/DA and K-OMG (50 random samples)
Overall Offensiveness Score4.196
2
Detoxification Dataset Quality EvaluationK/DA Ours En 500 neutral-toxic pairs Current Paper
Overall Quality Score2.717
1
Toxic-neutral pair quality evaluationK/DA
Overall Score2.719
1
Detoxification Dataset Quality EvaluationParaDetox 500 neutral-toxic pairs--
1
Detoxification Dataset Quality EvaluationToxiGen 500 neutral-toxic pairs--
1
Toxic-neutral pair quality evaluationK-OMG--
1
Toxic-neutral pair quality evaluationBEEP--
1
Showing 10 of 12 rows

Other info

Code

Follow for update