MeloDISinger: Melody-Aware & Duration-Preserving Singing Voice Editing with Audio Infilling
About
Text-based singing voice editing (SVE) aims to revise sung lyrics while preserving the original melody, total duration, and non-edited regions. In this paper, we propose MeloDISinger, a flow-matching-based SVE model for melody-aware and duration-preserving editing. Its core module, MeloDRP, predicts fixed-budget duration ratios, enabling explicit span-wise duration control. For melody-aware duration allocation, MeloDRP fuses phonetic cues with pseudo-MIDI melodic context through cross-attention, while temporal-overlap supervision encourages soft phoneme--note correspondences. We further use a flow-matching mel decoder for audio infilling to synthesize edited regions while preserving surrounding context. In addition, we introduce a duration-aware edited-lyric generation pipeline using WhisperX and an LLM to construct feasible evaluation scenarios. Experiments demonstrate state-of-the-art performance in both objective and subjective evaluations.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Singing Voice Editing | Singing Voice Editing Rep-P scenario | Lyric Following3.66 | 3 | |
| Singing Voice Editing | Singing Voice Editing Del scenario | Lyric Following4.21 | 3 | |
| Singing Voice Editing | GTSinger En (Rep-P) | WER31.33 | 3 | |
| Singing Voice Editing | GTSinger-En (Ins) | WER18.57 | 3 | |
| Singing Voice Editing | GTSinger-En (Del) | WER24.88 | 3 | |
| Singing Voice Editing | Singing Voice Editing Ins scenario | Lyric Following Score3.95 | 3 | |
| Singing Voice Editing | Singing Voice Editing Rep-S scenario | Lyric Following Score4.26 | 2 | |
| Singing Voice Editing | Singing Voice Editing Rep-SM scenario | Lyric Following4.05 | 2 | |
| Singing Voice Editing | Singing Voice Editing Mix scenario | Lyric Following4.12 | 2 | |
| Singing Voice Editing | GTSinger-En (Rep-S) | WER21.88 | 2 |