SwitchLight: Co-design of Physics-driven Architecture and Pre-training Framework for Human Portrait Relighting
About
We introduce a co-designed approach for human portrait relighting that combines a physics-guided architecture with a pre-training framework. Drawing on the Cook-Torrance reflectance model, we have meticulously configured the architecture design to precisely simulate light-surface interactions. Furthermore, to overcome the limitation of scarce high-quality lightstage data, we have developed a self-supervised pre-training strategy. This novel combination of accurate physical modeling and expanded training dataset establishes a new benchmark in relighting realism.
Hoon Kim, Minje Jang, Wonjun Yoon, Jisoo Lee, Donghyun Na, Sanghyun Woo• 2024
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Diffuse albedo prediction | FaceOLAT (test) | PSNR32.98 | 5 | |
| Video Relighting | BodyReLux Captured Performers Dataset Held-out sequences and views 1.0 (test) | PSNR16.88 | 5 | |
| Diffuse albedo prediction | 3DRFE | PSNR31.08 | 5 | |
| Portrait Relighting | POLAR | LPIPS0.168 | 4 |
Showing 4 of 4 rows