Our new X account is live! Follow @wizwand_team for updates
WorkDL logo mark

Epipolar-Free 3D Gaussian Splatting for Generalizable Novel View Synthesis

About

Generalizable 3D Gaussian splitting (3DGS) can reconstruct new scenes from sparse-view observations in a feed-forward inference manner, eliminating the need for scene-specific retraining required in conventional 3DGS. However, existing methods rely heavily on epipolar priors, which can be unreliable in complex realworld scenes, particularly in non-overlapping and occluded regions. In this paper, we propose eFreeSplat, an efficient feed-forward 3DGS-based model for generalizable novel view synthesis that operates independently of epipolar line constraints. To enhance multiview feature extraction with 3D perception, we employ a selfsupervised Vision Transformer (ViT) with cross-view completion pre-training on large-scale datasets. Additionally, we introduce an Iterative Cross-view Gaussians Alignment method to ensure consistent depth scales across different views. Our eFreeSplat represents an innovative approach for generalizable novel view synthesis. Different from the existing pure geometry-free methods, eFreeSplat focuses more on achieving epipolar-free feature matching and encoding by providing 3D priors through cross-view pretraining. We evaluate eFreeSplat on wide-baseline novel view synthesis tasks using the RealEstate10K and ACID datasets. Extensive experiments demonstrate that eFreeSplat surpasses state-of-the-art baselines that rely on epipolar priors, achieving superior geometry reconstruction and novel view synthesis quality. Project page: https://tatakai1.github.io/efreesplat/.

Zhiyuan Min, Yawei Luo, Jianwen Sun, Yi Yang• 2024

Related benchmarks

TaskDatasetResultRank
Novel View SynthesisRealEstate10K
PSNR26.45
116
Novel View SynthesisACID
PSNR28.3
51
Novel View SynthesisACID (test)
PSNR28.3
18
Novel View SynthesisRealEstate 10k (RE10k) (test)
PSNR26.45
16
Novel View SynthesisRealEstate10K overlap < 0.7
PSNR25.72
3
Novel View SynthesisRealEstate10K (overlap < 0.6)
PSNR25.48
3
Novel View SynthesisRealEstate10K overlap < 0.5
PSNR25.46
3
Showing 7 of 7 rows

Other info

Code

Follow for update