Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

FreeSplatter: Pose-free Gaussian Splatting for Sparse-view 3D Reconstruction

About

Sparse-view reconstruction models typically require precise camera poses, yet obtaining these parameters from sparse-view images remains challenging. We introduce FreeSplatter, a scalable feed-forward framework that generates high-quality 3D Gaussians from uncalibrated sparse-view images while estimating camera parameters within seconds. Our approach employs a streamlined transformer architecture where self-attention blocks facilitate information exchange among multi-view image tokens, decoding them into pixel-aligned 3D Gaussian primitives within a unified reference frame. This representation enables both high-fidelity 3D modeling and efficient camera parameter estimation using off-the-shelf solvers. We develop two specialized variants--for object-centric and scene-level reconstruction--trained on comprehensive datasets. Remarkably, FreeSplatter outperforms several pose-dependent Large Reconstruction Models (LRMs) by a notable margin while achieving comparable or even better pose estimation accuracy compared to state-of-the-art pose-free reconstruction approach MASt3R in challenging benchmarks. Beyond technical benchmarks, FreeSplatter streamlines text/image-to-3D content creation pipelines, eliminating the complexity of camera pose management while delivering exceptional visual fidelity.

Jiale Xu, Shenghua Gao, Ying Shan• 2024

Related benchmarks

TaskDatasetResultRank
Surface ReconstructionDTU
CD (Scan 24)5.673
43
Novel View SynthesisDTU 1 (test)
PSNR18.26
35
Surface ReconstructionReplica Office0, Office1, Office2, Office3, Office4, Room0, Room1, Room2
CD (mm) - Office0169.9
21
Amodal 3D object generationGSO
FID72.88
14
Novel View SynthesisBlendedMVS (test)
PSNR15.18
13
Novel View SynthesisReplica (test)
PSNR14.55
13
Camera pose estimationDTU
Rotational Error (°)38.271
6
Camera pose estimationReplica
Rotational Error (°)47.136
6
Showing 8 of 8 rows

Other info

Follow for update