Flow Matching Guide and Code
About
Flow Matching (FM) is a recent framework for generative modeling that has achieved state-of-the-art performance across various domains, including image, video, audio, speech, and biological structures. This guide offers a comprehensive and self-contained review of FM, covering its mathematical foundations, design choices, and extensions. By also providing a PyTorch package featuring relevant examples (e.g., image and text generation), this work aims to serve as a resource for both novice and experienced researchers interested in understanding, applying and further developing FM.
Yaron Lipman, Marton Havasi, Peter Holderrieth, Neta Shaul, Matt Le, Brian Karrer, Ricky T. Q. Chen, David Lopez-Paz, Heli Ben-Hamu, Itai Gat• 2024
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Trajectory Planning | Experiment Simulation 1 | Trajectory Time (s)5.77 | 7 | |
| Probabilistic Regression | Synthetic Manifold (test) | AUROC56.5 | 6 | |
| Trajectory Planning | Online replanning for object grasps Experiment 2 | Success Rate15 | 5 | |
| Surface Temperature Forecasting | ERA5 (test) | MSE0.0038 | 4 | |
| Cross-Domain Style Transfer | MNIST to SVHN | FID (50K)4.102 | 3 | |
| Conditional Generation | Synthetic Manifold (test) | AUROC0.51 | 3 | |
| Extrapolation Detection | MNIST to SVHN vs FMNIST | AUROC0.529 | 3 | |
| Extrapolation Detection | MNIST to SVHN vs KMNIST | AUROC0.516 | 3 | |
| Human-robot object handover | Human-human handover dataset (test) | Terminal Constraint Violations0.05 | 3 | |
| Physical Anomaly Detection | ERA5 (test) | AUROC0.556 | 3 |
Showing 10 of 10 rows