MagicBrush
Benchmarks
| Task Name | Dataset Name | SOTA Result | Trend | |
|---|---|---|---|---|
| Instructive image editing | MagicBrush (test) | CLIP Image0.9281 | 53 | |
| Image Manipulation Localization | MagicBrush | F1 Score81.95 | 21 | |
| Image Editing | MagicBrush | CLIPim91.1 | 19 | |
| Visual World Modelling | MagicBrush | GPT-4o Score8.14 | 18 | |
| Instruction-guided image editing | MagicBrush single-turn (test) | CLIP Similarity (Image)0.9332 | 13 | |
| Instructive Image Editing | MagicBrush 1.0 (test) | CLIP Image Similarity0.911 | 12 | |
| Local Image Editing | MagicBrush (dev) | L1 Error0.0463 | 12 | |
| Image Inpainting | MagicBrush | IR Score4.2 | 12 | |
| Image Editing | MagicBrush Single-Turn | L1 Loss0.033 | 11 | |
| Change Detection | MagicBrush | F1 Score89.9 | 10 | |
| Description-guided Image Editing | MagicBrush Multi-turn (test) | L1 Loss0.0911 | 10 | |
| Image-level forgery detection | MagicBrush | Accuracy90.1 | 9 | |
| Forward-dynamics Prediction | MagicBrush AURORA-BENCH | GPT-4o Score6.71 | 9 | |
| Instruction-based image editing | MagicBrush | L1 Loss0.0641 | 9 | |
| Image Editing | MagicBrush (test) | L1 Error0.06 | 9 | |
| instruction-based object addition | MagicBrush | CLIP Score0.9312 | 7 | |
| Image Editing | MagicBrush v1 (test) | CLIP Input Similarity0.898 | 7 | |
| Image Editing | MagicBrush Multi-Turn | L1 Loss0.035 | 7 | |
| Instruction-guided image editing | MagicBrush multi-turn (test) | CLIP-T0.313 | 7 | |
| Differences Caption Generation | MagicBrush (test) | Main Difference Count50 | 7 | |
| Edit Inspectors Question Answering | MagicBrush (test) | Accuracy73.7 | 7 | |
| Instruction-guided image editing | MagicBrush | LPIPS-U0.0446 | 6 | |
| Instruction-based image editing | MagicBrush All Turn (test) | L1 Loss0.043 | 4 | |
| Instruction-based image editing | MagicBrush Final Turn (test) | L10.075 | 4 |
Showing 24 of 24 rows