MuVieCAST: Multi-View Consistent Artistic Style Transfer
About
We introduce MuVieCAST, a modular multi-view consistent style transfer network architecture that enables consistent style transfer between multiple viewpoints of the same scene. This network architecture supports both sparse and dense views, making it versatile enough to handle a wide range of multi-view image datasets. The approach consists of three modules that perform specific tasks related to style transfer, namely content preservation, image transformation, and multi-view consistency enforcement. We extensively evaluate our approach across multiple application domains including depth-map-based point cloud fusion, mesh reconstruction, and novel-view synthesis. Our experiments reveal that the proposed framework achieves an exceptional generation of stylized images, exhibiting consistent outcomes across perspectives. A user study focusing on novel-view synthesis further confirms these results, with approximately 68\% of cases participants expressing a preference for our generated outputs compared to the recent state-of-the-art method. Our modular framework is extensible and can easily be integrated with various backbone architectures, making it a flexible solution for multi-view style transfer. More results are demonstrated on our project page: muviecast.github.io
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| 3D Scene Stylization | Horse greatwave | CHD0.3042 | 3 | |
| 3D Scene Stylization | Panther mosaic | CHD0.1238 | 3 | |
| 3D Scene Stylization | Lighthouse abstract | CHD0.3815 | 3 | |
| 3D Scene Stylization | Francis abstract2 | CHD0.1623 | 3 | |
| 3D Scene Stylization | starry (train) | CHD0.1538 | 3 | |
| SLAM-based 3D consistency | Horse greatwave | ATE0.058 | 3 | |
| SLAM-based 3D consistency | Panther mosaic | ATE0.005 | 3 | |
| SLAM-based 3D consistency | Lighthouse abstract | ATE1.081 | 3 | |
| SLAM-based 3D consistency | Francis abstract2 | ATE0.752 | 3 | |
| SLAM-based 3D consistency | starry (train) | ATE0.81 | 3 |