Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

FlowPolicy: Enabling Fast and Robust 3D Flow-based Policy via Consistency Flow Matching for Robot Manipulation

About

Robots can acquire complex manipulation skills by learning policies from expert demonstrations, which is often known as vision-based imitation learning. Generating policies based on diffusion and flow matching models has been shown to be effective, particularly in robotic manipulation tasks. However, recursion-based approaches are inference inefficient in working from noise distributions to policy distributions, posing a challenging trade-off between efficiency and quality. This motivates us to propose FlowPolicy, a novel framework for fast policy generation based on consistency flow matching and 3D vision. Our approach refines the flow dynamics by normalizing the self-consistency of the velocity field, enabling the model to derive task execution policies in a single inference step. Specifically, FlowPolicy conditions on the observed 3D point cloud, where consistency flow matching directly defines straight-line flows from different time states to the same action space, while simultaneously constraining their velocity values, that is, we approximate the trajectories from noise to robot actions by normalizing the self-consistency of the velocity field within the action space, thus improving the inference efficiency. We validate the effectiveness of FlowPolicy in Adroit and Metaworld, demonstrating a 7$\times$ increase in inference speed while maintaining competitive average success rates compared to state-of-the-art methods. Code is available at https://github.com/zql-kk/FlowPolicy.

Qinglun Zhang, Zhen Liu, Haoqiang Fan, Guanghui Liu, Bing Zeng, Shuaicheng Liu• 2024

Related benchmarks

TaskDatasetResultRank
Robot ManipulationMetaWorld 50 tasks
Success Rate (Easy)85
21
Robot ManipulationMetaWorld Medium 11 tasks
Success Rate73.6
18
Robot ManipulationMetaWorld Hard (6 tasks)
Success Rate46.2
18
Robot ManipulationAdroit
Success Rate69.3
18
Robot ManipulationMetaWorld Very Hard 5 tasks
Success Rate80
15
Robot ManipulationMeta-World
Latency (Easy) (ms)12
15
Robotic Arm ManipulationMetaWorld Easy
Success Rate92.1
15
Robotic Arm ManipulationMetaWorld Very Hard
Success Rate80
15
Robot ManipulationDexArt
Success Rate81
14
Robotic ManipulationAdroit and MetaWorld
Average Success Rate77.2
13
Showing 10 of 37 rows

Other info

Follow for update