Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

MeloDISinger: Melody-Aware & Duration-Preserving Singing Voice Editing with Audio Infilling

About

Text-based singing voice editing (SVE) aims to revise sung lyrics while preserving the original melody, total duration, and non-edited regions. In this paper, we propose MeloDISinger, a flow-matching-based SVE model for melody-aware and duration-preserving editing. Its core module, MeloDRP, predicts fixed-budget duration ratios, enabling explicit span-wise duration control. For melody-aware duration allocation, MeloDRP fuses phonetic cues with pseudo-MIDI melodic context through cross-attention, while temporal-overlap supervision encourages soft phoneme--note correspondences. We further use a flow-matching mel decoder for audio infilling to synthesize edited regions while preserving surrounding context. In addition, we introduce a duration-aware edited-lyric generation pipeline using WhisperX and an LLM to construct feasible evaluation scenarios. Experiments demonstrate state-of-the-art performance in both objective and subjective evaluations.

Yoonjeong Park, Jaekwon Im, Juhan Nam• 2026

Related benchmarks

TaskDatasetResultRank
Singing Voice EditingSinging Voice Editing Rep-P scenario
Lyric Following3.66
3
Singing Voice EditingSinging Voice Editing Del scenario
Lyric Following4.21
3
Singing Voice EditingGTSinger En (Rep-P)
WER31.33
3
Singing Voice EditingGTSinger-En (Ins)
WER18.57
3
Singing Voice EditingGTSinger-En (Del)
WER24.88
3
Singing Voice EditingSinging Voice Editing Ins scenario
Lyric Following Score3.95
3
Singing Voice EditingSinging Voice Editing Rep-S scenario
Lyric Following Score4.26
2
Singing Voice EditingSinging Voice Editing Rep-SM scenario
Lyric Following4.05
2
Singing Voice EditingSinging Voice Editing Mix scenario
Lyric Following4.12
2
Singing Voice EditingGTSinger-En (Rep-S)
WER21.88
2
Showing 10 of 12 rows

Other info

Follow for update