Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

TeMuDance: Contrastive Alignment-Based Textual Control for Music-Driven Dance Generation

About

Existing music-driven dance generation approaches have achieved strong realism and effective audio-motion alignment. However, they generally lack semantic controllability, making it difficult to guide specific movements through natural language descriptions. This limitation primarily stems from the absence of large-scale datasets that jointly align music, text, and motion for supervised learning of text-conditioned control. To address this challenge, we propose TeMuDance, a framework that enables text-based control for music-conditioned dance generation without requiring any manually annotated music-text-motion triplet dataset. TeMuDance introduces a motion-centred bridging paradigm that leverages motion as a shared semantic anchor to align disjoint music-dance and text-motion datasets within a unified embedding space, enabling cross-modal retrieval of missing modalities for end-to-end training. A lightweight text control branch is then trained on top of a frozen music-to-dance diffusion backbone, preserving rhythmic fidelity while enabling fine-grained semantic guidance. To further suppress noise inherent in the retrieved supervision, we design a dual-stream fine-tuning strategy with confidence-based filtering. We also propose a novel task-aligned metric that quantifies whether textual prompts induce the intended kinematic attributes under music conditioning. Extensive experiments demonstrate that TeMuDance achieves competitive dance quality while substantially improving text-conditioned control over existing methods.

Xinran Liu, Diptesh Kanojia, Wenwu Wang, Zhenhua Feng• 2026

Related benchmarks

TaskDatasetResultRank
Music-driven Dance GenerationFineDance (test)
BAS Score0.2342
38
Music-to-Dance GenerationAIST++
Beat Alignment0.2427
12
Music-to-Dance GenerationUser Study 12 music clips (test)
Ours Win Rate85.4
4
Text-Driven ControllabilityUser Study 8 action instructions (test)
Choreographic Coherence75
1
Showing 4 of 4 rows

Other info

Follow for update