Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

Learn2Splat: Extending the Horizon of Learned 3DGS Optimization

About

3D Gaussian Splatting (3DGS) optimization is most commonly performed using standard optimizers (Adam, SGD). While stable across diverse scenes, standard optimizers are general-purpose and not tailored to the structure of the problem. In particular, they produce independent parameter updates that do not capture the structural and spatial relationships within a scene, leading to inefficient optimization and slow convergence. Recent works introduced learned optimizers that predict correlated updates informed by inter-parameter and inter-Gaussian dependencies. However, these methods are trained for a fixed number of optimization iterations and rely on manually scheduled learning rates to avoid degradation. In this paper, we introduce a learned optimizer for 3DGS that avoids degradation over extended optimization horizons without auxiliary mechanisms. To enable this, we propose a meta-learning scheme that extends the optimization horizon via a checkpoint buffer and an optimizer rollout strategy, combined with an architecture that encodes gradient scale information in its latent states. Results show improved early novel view synthesis quality while remaining stable over long horizons, with zero-shot generalization to unseen reconstruction settings. To support our findings, we introduce the first unified framework for training and evaluating both learned and conventional optimizers across sparse and dense view settings. Code and models will be released publicly. Our project page is available at https://naamapearl.github.io/learn2splat .

Naama Pearl, Stefano Esposito, Haofei Xu, Amit Peleg, Patricia Gschossmann, Lorenzo Porzi, Peter Kontschieder, Gerard Pons-Moll, Andreas Geiger• 2026

Related benchmarks

TaskDatasetResultRank
Novel View SynthesisMip-NeRF360 (test)
PSNR25.69
80
Novel View SynthesisDL3DV Zero-shot 8 views, 512 x 960
PSNR27.28
10
3D ReconstructionDL3DV Zero-shot 32 views, 256 x 448
PSNR30.21
9
Novel View SynthesisRealEstate10K 8 views, 512 x 960 (test)
PSNR28.68
6
Novel View SynthesisLLFF Dense (~20 to 60 views, 756×1008) (test)
PSNR24.47
4
Novel View SynthesisDL3DV In-domain 100+ views 256 x 480
PSNR28.89
4
Novel View SynthesisDL3DV 100+ views 256 x 480 (In-domain)
PSNR28.89
4
Novel View SynthesisLLFF Zero-shot ~ 20 to 60 views, 756 x 1008
PSNR24.47
4
Novel View SynthesisMip-NeRF360 Zero-shot 100+ views, 520 x 780
PSNR25.69
4
Novel View SynthesisDTU Zero-shot ~ 30 views, 1162 x 1554
PSNR29.1
4
Showing 10 of 10 rows

Other info

Follow for update