Share your thoughts, 1 month free Claude Pro on usSee more
WorkDL logo mark

DLGStream: Dynamic Language-embedded Guassian Splatting for Open-vocabulary Enabled Free-viewpoint Video Streaming

About

3D Gaussian Splatting~(3DGS) has emerged as a promising paradigm for reconstructing streamable free-viewpoint video~(FVV) from multi-view videos. However, 3DGS-based FVVs typically lack user interaction and editing capabilities, which diminishes the immersive experience. Recent research has integrated language features from CLIP into 3DGS via distillation, enabling open-vocabulary queries and supporting many downstream applications. Nevertheless, the stringent requirements of FVV, low frame size and high FPS, make current language Gaussian representations unsuitable for language-embedded FVV. In this paper, we propose DLGStream, a novel language-embedded FVV representation that streams time-varying language features alongside Gaussian attributes to support 4D environment interaction, scene editing, and spatial intelligence. Specifically, we propose a dual-opacity dynamic language Gaussian representation, which maintains two opacity attributes for color and language features to deal with performance degradation that occurs when colors and features are jointly optimized. Furthermore, we introduce an interpolation-based deformation field to reduce temporal redundancy. This deformation field can also be used for 4D frame interpolation, boosting FVV sequences from low to high FPS. Experimental results demonstrate that DLGStream achieves superior performance in both on open-vocabulary segmentation and reconstruction quality with an average frame size of merely 43 KB. The code is available on \href{https://github.com/kkkzh/DLGStream}{https://github.com/kkkzh/DLGStream}.

Zhihui Ke, Yuyang Liu, Xiaobo Zhou, Tie Qiu• 2026

Related benchmarks

TaskDatasetResultRank
Dynamic Scene ReconstructionN3DV (test)
PSNR32.26
45
Dynamic Scene ReconstructionN3DV
PSNR (dB)32.27
28
FVV reconstructionN3DV
PSNR32.26
17
Dynamic Scene ReconstructionMeetRoom
PSNR27.48
9
FVV reconstructionMeetRoom
PSNR27.48
9
Dynamic Scene ReconstructionN3DV Flame Salmon (long video sequence)
PSNR29.08
6
Free-viewpoint Video ReconstructionWideRange4D
PSNR27.58
2
Showing 7 of 7 rows

Other info

Follow for update