AuraFusion360: Augmented Unseen Region Alignment for Reference-based 360{\deg} Unbounded Scene Inpainting
About
Three-dimensional scene inpainting is crucial for applications from virtual reality to architectural visualization, yet existing methods struggle with view consistency and geometric accuracy in 360{\deg} unbounded scenes. We present AuraFusion360, a novel reference-based method that enables high-quality object removal and hole filling in 3D scenes represented by Gaussian Splatting. Our approach introduces (1) depth-aware unseen mask generation for accurate occlusion identification, (2) Adaptive Guided Depth Diffusion, a zero-shot method for accurate initial point placement without requiring additional training, and (3) SDEdit-based detail enhancement for multi-view coherence. We also introduce 360-USID, the first comprehensive dataset for 360{\deg} unbounded scene inpainting with ground truth. Extensive experiments demonstrate that AuraFusion360 significantly outperforms existing methods, achieving superior perceptual quality while maintaining geometric accuracy across dramatic viewpoint changes.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| 3D Object Removal | 360-USID Plant | PSNR18.001 | 9 | |
| 3D Object Removal | 360-USID Skateboard | PSNR17.007 | 9 | |
| 3D Object Removal | 360-USID Sunflower | PSNR24.943 | 9 | |
| 3D Object Removal | 360-USID Carton | PSNR17.675 | 9 | |
| 3D Object Removal | 360-USID Cookie | PSNR12.841 | 9 | |
| 3D Object Removal | 360-USID Newcone | PSNR17.536 | 9 | |
| 3D Object Removal | 360-USID Average | PSNR17.661 | 9 | |
| 3D Object Removal | 360-USID Cone | PSNR15.626 | 9 |