Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

RePlan: Reasoning-guided Region Planning for Complex Instruction-based Image Editing

About

Instruction-based image editing enables natural-language control over visual modifications, yet existing models falter under Instruction-Visual Complexity (IV-Complexity), where intricate instructions meet cluttered or ambiguous scenes. We introduce RePlan (Region-aligned Planning), a plan-then-execute framework that couples a vision-language planner with a diffusion editor. The planner decomposes instructions via step-by-step reasoning and explicitly grounds them to target regions; the editor then applies changes using a training-free attention-region injection mechanism, enabling precise, parallel multi-region edits without iterative inpainting. To strengthen planning, we apply GRPO-based reinforcement learning using 1K instruction-only examples, yielding substantial gains in reasoning fidelity and format reliability. We further present IV-Edit, a benchmark focused on fine-grained grounding and knowledge-intensive edits. Across IV-Complex settings, RePlan consistently outperforms strong baselines trained on far larger datasets, improving regional precision and overall consistency. Our project page: https://replan-iv-edit.github.io

Tianyuan Qu, Lei Ke, Xiaohang Zhan, Longxiang Tang, Yuqi Liu, Bohao Peng, Bei Yu, Dong Yu, Jiaya Jia• 2025

Related benchmarks

TaskDatasetResultRank
Image EditingProductConsistency (test)
Character Error Rate (CER)0.2914
17
Image EditingVI-Edit (test)
Quality Score4.16
9
Image EditingRePlan
Quality3.82
9
Image EditingImgEdit Easy
Add Score3.59
9
Image EditingImgEdit Hard
Average Score3.52
9
Image EditingPICA
Average Score50.18
9
Image EditingIVEdit (test)
Quality Score3.86
8
Showing 7 of 7 rows

Other info

GitHub

Follow for update