POp-GS: Next Best View in 3D-Gaussian Splatting with P-Optimality
About
In this paper, we present a novel algorithm for quantifying uncertainty and information gained within 3D Gaussian Splatting (3D-GS) through P-Optimality. While 3D-GS has proven to be a useful world model with high-quality rasterizations, it does not natively quantify uncertainty or information, posing a challenge for real-world applications such as 3D-GS SLAM. We propose to quantify information gain in 3D-GS by reformulating the problem through the lens of optimal experimental design, which is a classical solution widely used in literature. By restructuring information quantification of 3D-GS through optimal experimental design, we arrive at multiple solutions, of which T-Optimality and D-Optimality perform the best quantitatively and qualitatively as measured on two popular datasets. Additionally, we propose a block diagonal covariance approximation which provides a measure of correlation at the expense of a greater computation cost.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Whole Scene Reconstruction | BlenderProc Synthetic Dataset 10 Scenes | PSNR26.69 | 17 | |
| Whole Scene Reconstruction | Captured Scene Manual SAM Mask (2 scenes) | PSNR15.65 | 17 | |
| Whole Scene Reconstruction | Captured Scene Grounded SAM2 Mask (2 scenes) | PSNR16.29 | 17 | |
| Whole Scene Reconstruction | GraspNet Real (10 Scenes) | PSNR21.16 | 17 | |
| Batch View Selection | Blender (test) | PSNR27.89 | 7 | |
| Batch View Selection | Mip-NeRF360 (test) | PSNR20.86 | 7 | |
| Keyframe Selection | Mip-NeRF360 (test) | PSNR18.73 | 7 | |
| Single View Selection | Mip-NeRF360 average over nine scenes 10 views | PSNR18.15 | 7 | |
| Single View Selection | Blender Dataset (test) | PSNR25.52 | 7 | |
| Single View Selection | Mip-Nerf360 20 views | PSNR21.32 | 7 |