Trust Region Policy Optimization
About
We describe an iterative procedure for optimizing policies, with guaranteed monotonic improvement. By making several approximations to the theoretically-justified procedure, we develop a practical algorithm, called Trust Region Policy Optimization (TRPO). This algorithm is similar to natural policy gradient methods and is effective for optimizing large nonlinear policies such as neural networks. Our experiments demonstrate its robust performance on a wide variety of tasks: learning simulated robotic swimming, hopping, and walking gaits; and playing Atari games using images of the screen as input. Despite its approximations that deviate from the theory, TRPO tends to give monotonic improvement, with little tuning of hyperparameters.
John Schulman, Sergey Levine, Philipp Moritz, Michael I. Jordan, Pieter Abbeel• 2015
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Reinforcement Learning | Walker | Average Returns229.8 | 38 | |
| Quadruped | Quadruped | Return206.2 | 33 | |
| Reinforcement Learning | Humanoid | Zero-Shot Reward3.18e+3 | 30 | |
| Reinforcement Learning | Trading | Return9.17 | 24 | |
| Reinforcement Learning | cheetah | Return210.1 | 24 | |
| Reinforcement Learning | LunarLander v2 | Final Return104 | 23 | |
| Reinforcement Learning | Halfcheetah | Average Return1.77e+3 | 17 | |
| Robot Locomotion | Humanoid | Cumulative Reward1.39 | 16 | |
| Reinforcement Learning | Pendulum | Avg Episode Reward-145.5 | 15 | |
| Reinforcement Learning | Hopper | Avg Episode Reward2.59e+3 | 15 |
Showing 10 of 69 rows