FreeSplatter: Pose-free Gaussian Splatting for Sparse-view 3D Reconstruction
About
Sparse-view reconstruction models typically require precise camera poses, yet obtaining these parameters from sparse-view images remains challenging. We introduce FreeSplatter, a scalable feed-forward framework that generates high-quality 3D Gaussians from uncalibrated sparse-view images while estimating camera parameters within seconds. Our approach employs a streamlined transformer architecture where self-attention blocks facilitate information exchange among multi-view image tokens, decoding them into pixel-aligned 3D Gaussian primitives within a unified reference frame. This representation enables both high-fidelity 3D modeling and efficient camera parameter estimation using off-the-shelf solvers. We develop two specialized variants--for object-centric and scene-level reconstruction--trained on comprehensive datasets. Remarkably, FreeSplatter outperforms several pose-dependent Large Reconstruction Models (LRMs) by a notable margin while achieving comparable or even better pose estimation accuracy compared to state-of-the-art pose-free reconstruction approach MASt3R in challenging benchmarks. Beyond technical benchmarks, FreeSplatter streamlines text/image-to-3D content creation pipelines, eliminating the complexity of camera pose management while delivering exceptional visual fidelity.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Surface Reconstruction | DTU | CD (Scan 24)5.673 | 43 | |
| Novel View Synthesis | DTU 1 (test) | PSNR18.26 | 35 | |
| Surface Reconstruction | Replica Office0, Office1, Office2, Office3, Office4, Room0, Room1, Room2 | CD (mm) - Office0169.9 | 21 | |
| Amodal 3D object generation | GSO | FID72.88 | 14 | |
| Novel View Synthesis | BlendedMVS (test) | PSNR15.18 | 13 | |
| Novel View Synthesis | Replica (test) | PSNR14.55 | 13 | |
| Camera pose estimation | DTU | Rotational Error (°)38.271 | 6 | |
| Camera pose estimation | Replica | Rotational Error (°)47.136 | 6 |