Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Bilingual Text-to-Motion Generation: A New Benchmark and Baselines

About

Text-to-motion generation holds significant potential for cross-linguistic applications, yet it is hindered by the lack of bilingual datasets and the poor cross-lingual semantic understanding of existing language models. To address these gaps, we introduce BiHumanML3D, the first bilingual text-to-motion benchmark, constructed via LLM-assisted annotation and rigorous manual correction. Furthermore, we propose a simple yet effective baseline, Bilingual Motion Diffusion (BiMD), featuring Cross-Lingual Alignment (CLA). CLA explicitly aligns semantic representations across languages, creating a robust conditional space that enables high-quality motion generation from bilingual inputs, including zero-shot code-switching scenarios. Extensive experiments demonstrate that BiMD with CLA achieves an FID of 0.045 vs. 0.169 and R@3 of 82.8\% vs. 80.8\%, significantly outperforms monolingual diffusion models and translation baselines on BiHumanML3D, underscoring the critical necessity and reliability of our dataset and the effectiveness of our alignment strategy for cross-lingual motion synthesis. The dataset and code are released at \href{https://wengwanjiang.github.io/BilingualT2M-page}{https://wengwanjiang.github.io/BilingualT2M-page}

Wanjiang Weng, Xiaofeng Tan, Xiangbo Shu, Guo-Sen Xie, Pan Zhou, Hongsong Wang• 2026

Related benchmarks

TaskDatasetResultRank
Text-to-motion generationHumanML3D (test)
FID0.045
481
Text-to-motion generationBiHumanML3D English
R-Precision (Top 1)55.8
9
Text-to-motion generationBiHumanML3D Chinese
R Precision (Top 1)54.2
9
Text-to-motion generationHumanML3D English prompts
Win Rate35.4
3
Showing 4 of 4 rows

Other info

Follow for update