Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

FlowPolicy: Enabling Fast and Robust 3D Flow-based Policy via Consistency Flow Matching for Robot Manipulation

About

Robots can acquire complex manipulation skills by learning policies from expert demonstrations, which is often known as vision-based imitation learning. Generating policies based on diffusion and flow matching models has been shown to be effective, particularly in robotic manipulation tasks. However, recursion-based approaches are inference inefficient in working from noise distributions to policy distributions, posing a challenging trade-off between efficiency and quality. This motivates us to propose FlowPolicy, a novel framework for fast policy generation based on consistency flow matching and 3D vision. Our approach refines the flow dynamics by normalizing the self-consistency of the velocity field, enabling the model to derive task execution policies in a single inference step. Specifically, FlowPolicy conditions on the observed 3D point cloud, where consistency flow matching directly defines straight-line flows from different time states to the same action space, while simultaneously constraining their velocity values, that is, we approximate the trajectories from noise to robot actions by normalizing the self-consistency of the velocity field within the action space, thus improving the inference efficiency. We validate the effectiveness of FlowPolicy in Adroit and Metaworld, demonstrating a 7$\times$ increase in inference speed while maintaining competitive average success rates compared to state-of-the-art methods. Code is available at https://github.com/zql-kk/FlowPolicy.

Qinglun Zhang, Zhen Liu, Haoqiang Fan, Guanghui Liu, Bing Zeng, Shuaicheng Liu• 2024

Related benchmarks

TaskDatasetResultRank
Robot ManipulationAdroit
Pen Task Score58
50
Robotic ManipulationAdroit and MetaWorld
Average Success Rate77.2
21
Robot ManipulationMetaWorld 50 tasks
Success Rate (Easy)85
21
Dexterous Hand ControlAdroit
Overall Avg Success Rate70
19
Robot ManipulationMetaWorld Medium 11 tasks
Success Rate73.6
18
Robot ManipulationMetaWorld Hard (6 tasks)
Success Rate46.2
18
Robot ManipulationAdroit
Success Rate69.3
18
3D pointcloud manipulationMetaWorld
Success Rate (Easy)84.8
17
Robotic ManipulationMeta-World
Success Rate (Easy)84.3
16
Robot ManipulationMetaWorld Very Hard 5 tasks
Success Rate80
15
Showing 10 of 45 rows

Other info

Follow for update