Learning 4D Geometric Priors for Inference-Efficient World Action Models
About
World Action Models (WAMs) have shown strong potential for robotic manipulation by jointly modeling visual future dynamics and executable action sequences. However, existing video-action co-training methods primarily optimize appearance-oriented video latents, which may insufficiently capture the temporally evolving geometry required for precise manipulation. We propose MECo-WAM, a Multi-Expert Co-Training World Action Model that injects action-relevant 4D geometric priors into video-action representations while preserving the original lightweight inference graph. During training, MECo-WAM combines video and action experts with a lightweight 4D expert supervised by relational targets from a frozen VGGT encoder. Asymmetric expert visibility prevents non-causal shortcuts from auxiliary geometry to action generation. To transfer geometric knowledge into the deployed video-action pathway, we introduce decayed 4D read-mask attention, which provides restricted current-frame geometric guidance early in training and progressively removes this dependency. We further propose action-aware temporal geometric distillation, which aligns within-frame geometric relations and their temporal evolution while emphasizing visual regions most relevant to robot actions. At deployment, all auxiliary 4D components are removed. Experiments on LIBERO (98.2%), RoboTwin 2.0 (92.6%), and challenging real-world manipulation tasks show that MECo-WAM improves manipulation performance without increasing inference cost.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Robot Manipulation | RoboTwin Randomized 2.0 | Overall Success Rate91.98 | 100 | |
| Robot Manipulation | RoboTwin Clean 2.0 | Success Rate93.26 | 74 | |
| Robotic Manipulation | RoboTwin 2.0 (Average) | Success Rate92.62 | 25 | |
| Object Manipulation | LIBERO | Spatial Success Rate98.8 | 11 | |
| Cube Sorting | Real-world tabletop Cube Sorting Avg | Success Rate70 | 3 | |
| Cube Sorting | Real-world tabletop Cube Sorting 2 (test) | Success Rate (SR)100 | 3 | |
| Cube Stacking | Real-world tabletop Cube Stacking (test 1) | Success Rate (%)100 | 3 | |
| Cube Stacking | Real-world tabletop Cube Stacking (test) | Success Rate (%)100 | 3 | |
| Cube Stacking | Real-world tabletop Cube Stacking Avg | Success Rate60 | 3 | |
| Cube Sorting | Real-world tabletop Cube Sorting (test 1) | Success Rate (SR)100 | 3 |