MangaNinja: Line Art Colorization with Precise Reference Following
About
Derived from diffusion models, MangaNinjia specializes in the task of reference-guided line art colorization. We incorporate two thoughtful designs to ensure precise character detail transcription, including a patch shuffling module to facilitate correspondence learning between the reference color image and the target line art, and a point-driven control scheme to enable fine-grained color matching. Experiments on a self-collected benchmark demonstrate the superiority of our model over current solutions in terms of precise colorization. We further showcase the potential of the proposed interactive point control in handling challenging cases, cross-character colorization, multi-reference harmonization, beyond the reach of existing algorithms.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Lineart Colorization | Lineart Colorization 900 samples (test) | Image Alignment Score90.25 | 23 | |
| Image-referenced Sketch Colorization | Triplet 50K (val) | FID42.85 | 7 | |
| Local Colourisation | Place365 Indoor | FID134.6 | 5 | |
| Local Colourisation | Place365 Outdoor | FID127.7 | 5 | |
| Local Colourisation | PascalVOC 2012 | FID289.2 | 5 | |
| Local Colourisation | Danbooru 2023 | FID304.2 | 5 | |
| Image Colorization | Animation Film Image Pairs 1.5K (test) | SSIM54.3 | 4 |