Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

AnyMo: Scaling Any-Modality Conditional Motion Generation with Masked Modeling

About

Conditional human motion generation remains a fundamental challenge in computer vision and robotics. Despite significant progress, current methods are often constrained by fixed modality configurations and task-specific architectures, leaving cross-modal interactions and the scaling laws of multimodal-conditioned synthesis largely underexplored. A key bottleneck is the scarcity of large-scale modality-aligned motion data, limiting generalization across diverse control signals. In this work, we introduce OmniHuMo, a large-scale, high-quality dataset comprising over 5,000 hours of motion and 3.2 million sequences with precisely aligned multimodal annotations (e.g., text, speech, music, and trajectory). Leveraging OmniHuMo, we propose AnyMo, a unified multimodal framework combining a Residual FSQ-based motion tokenizer with a scalable masked modeling transformer, enabling high-quality motion synthesis under arbitrary modality combinations. Extensive experiments show that AnyMo achieves high-fidelity synthesis while offering flexible control over both spatial and stylistic attributes.

Yiheng Li, Zhuo Li, Ruibing Hou, Yingjie Chen, Hong Chang, Hao Liu, Shiguang Shan• 2026

Related benchmarks

TaskDatasetResultRank
Motion ReconstructionHumanML3D (test)
MPJPE27.9
12
Motion ReconstructionMotionMillion (test)
MPJPE (mm)21.5
3
Motion ReconstructionOmniHuMo (test)
MPJPE13.2
2
Showing 3 of 3 rows

Other info

Follow for update