Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Odoriko: A Shape-Aware Multimodal Diffusion Framework for Human Motion

About

Human motion generation has been widely studied across diverse input modalities, text, music, and video, and recent efforts have unified these into single multimodal frameworks. However, while morphological factors such as gender and body shape are known to produce distinct kinematic signatures, no existing unified framework incorporates this into generation, treating all subjects as morphologically equivalent. We present Odoriko, the first unified multimodal motion generation framework that reflects subject bio-morphological information directly in synthesized motion output. Rather than averaging over subject variation, Odoriko generates motion that is consistent with who is moving, not just what they are asked to do, across text, music, and video conditions within a single model. When explicit morphological information is unavailable, Odoriko additionally recovers subject morphology alongside motion, unifying estimation and generation in one framework. Extensive experiments across text-to-motion, music-to-dance, and video-to-motion benchmarks demonstrate that Odoriko matches or exceeds prior specialized models on standard metrics, while enabling morphology-consistent generation that no existing unified framework supports.

Dongseok Shim, Julian Tanke, Kengo Uchida, Christian Simon, Koichi Saito, Takashi Shibuya, Shusuke Takahashi, Yuki Mitsufuji• 2026

Related benchmarks

TaskDatasetResultRank
3D Human Pose Estimation3DPW (test)
PA-MPJPE41.1
527
text-to-motion mappingHumanML3D (test)
FID0.103
298
Music-driven Dance GenerationFineDance (test)
BAS Score0.2496
49
Camera-space human motion estimationEMDB-1 (test)
PA-MPJPE41.1
8
Showing 4 of 4 rows

Other info

Follow for update