pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis
About
We have witnessed rapid progress on 3D-aware image synthesis, leveraging recent advances in generative visual models and neural rendering. Existing approaches however fall short in two ways: first, they may lack an underlying 3D representation or rely on view-inconsistent rendering, hence synthesizing images that are not multi-view consistent; second, they often depend upon representation network architectures that are not expressive enough, and their results thus lack in image quality. We propose a novel generative model, named Periodic Implicit Generative Adversarial Networks ($\pi$-GAN or pi-GAN), for high-quality 3D-aware image synthesis. $\pi$-GAN leverages neural representations with periodic activation functions and volumetric rendering to represent scenes as view-consistent 3D representations with fine detail. The proposed approach obtains state-of-the-art results for 3D-aware image synthesis with multiple real and synthetic datasets.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Unconditional image synthesis | FFHQ 256x256 (test) | FID83 | 31 | |
| Image Generation | FFHQ 256x256 (train) | FID55.2 | 20 | |
| Image Synthesis | FFHQ | FID85 | 16 | |
| Image Generation | CelebA 128x128 (test) | FID14.7 | 14 | |
| Rendering | FFHQ | Total Rendering Time (ms)154 | 13 | |
| Surface Reconstruction | BFM (test) | SIDE0.727 | 12 | |
| Unconditional image synthesis | AFHQ 256x256 (test) | FID47 | 12 | |
| 3D-aware Image Synthesis | FFHQ | FID28.1 | 10 | |
| Image Generation | CARLA 128 x 128 (test) | FID29.2 | 9 | |
| 3D-aware Image Synthesis | Cats (test) | FID16.6 | 9 |