Training-free Controllable Human Motion Generation under Heterogeneous Constraints
About
Training-free controllable motion generation has attracted growing interest for enabling flexible constraint enforcement without constraint-specific training. However, existing training-free methods require constraints to be continuous objective-based with differentiable losses, while many real-world requirements are criterion-based and provide only discontinuous, sparse, or even black-box feedback. In this paper, we propose Motion-Inference-as-Control (MIC), the first training-free motion generation framework that handles both continuous objective-based and criterion-based motion constraints under a shared mechanism. The key idea is to cast diffusion-based motion generation as a stochastic control problem. This perspective not only provides principled and practically effective step-wise control laws that support criterion-based constraints without requiring differentiability and naturally accommodate objective-based constraints as a special case, but also motivates a control-oriented constraint coordination mechanism that adaptively balances and reconciles motion constraints during generation. Experiments across diverse constraint settings demonstrate the effectiveness of our framework.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Human-Object Interaction Controllable Motion Generation | HOI-1 Open-set constraint specifications | Skating Score3.6 | 9 | |
| Geometric-constraint Controllable Motion Generation | Open-set constraint specifications GEO-1 | Skating Accuracy10.2 | 9 | |
| Controllable Motion Generation (HSI-1) | HumanML3D (test) | Skating Rate7.4 | 8 | |
| Human-Scene Interaction Controllable Motion Generation | Open-set constraint specifications HSI-3 | Skating0.112 | 7 | |
| Human-Scene Interaction Controllable Motion Generation | HSI-2 Open-set | Skating Accuracy17.2 | 7 |