Diffusion Model Predictive Control
About
We propose Diffusion Model Predictive Control (D-MPC), a novel MPC approach that learns a multi-step action proposal and a multi-step dynamics model, both using diffusion models, and combines them for use in online MPC. On the popular D4RL benchmark, we show performance that is significantly better than existing model-based offline planning methods using MPC (e.g. MBOP) and competitive with state-of-the-art (SOTA) model-based and model-free reinforcement learning methods. We additionally illustrate D-MPC's ability to optimize novel reward functions at run time and adapt to novel dynamics, and highlight its advantages compared to existing diffusion-based planning baselines.
Guangyao Zhou, Sivaramakrishnan Swaminathan, Rajkumar Vasudeva Raju, J. Swaroop Guntupalli, Wolfgang Lehrach, Joseph Ortiz, Antoine Dedieu, Miguel L\'azaro-Gredilla, Kevin Murphy• 2024
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Offline Reinforcement Learning | D4RL Medium-Replay Hopper | Normalized Score92.5 | 97 | |
| Offline Reinforcement Learning | D4RL Medium HalfCheetah | Normalized Score46 | 97 | |
| Offline Reinforcement Learning | D4RL Medium Walker2d | Normalized Score76.2 | 96 | |
| Offline Reinforcement Learning | D4RL Medium-Replay HalfCheetah | Normalized Score41.1 | 84 | |
| Offline Reinforcement Learning | D4RL Medium Hopper | Normalized Score61.2 | 64 | |
| Offline Reinforcement Learning | D4RL Medium-Replay Walker2d | Normalized Score78.8 | 42 | |
| Offline Reinforcement Learning | puzzle-4x4-play OGBench 5 tasks v0 | Average Success Rate0.00e+0 | 28 | |
| Offline Reinforcement Learning | scene-play OGBench 5 tasks v0 | Average Success Rate4 | 26 | |
| Offline Reinforcement Learning | cube-double-play OGBench 5 tasks v0 | Average Success Rate3 | 19 | |
| Offline Reinforcement Learning | puzzle-3x3-play OGBench 5 tasks v0 | Average Success Rate1 | 19 |
Showing 10 of 19 rows