Space-time Gaussian Surfels for High-Fidelity Dynamic Objects Segmentation and Representation

We introduce ST-ObjGS, a method using Space-time Gaussian surfels for accurate object segmentation within 4D representations. Our approach addresses the limitations of current Gaussian-based methods, which primarily focus on static 3D scene understanding and struggle with geometrically accurate object segmentation in complex dynamic scenes. To ensure robust object-level segmentation, we first integrate Grounded SAM 2, which enables text prompt-based object selection and tracking. We then learn a set of Gaussian surfels for object geometry representation and employ a marginal 1D Gaussian for dynamic modeling at each timestamp. To improve geometric quality when modeling surfaces, we use depth and surface normal for geometric regularization. Furthermore, to address continuity and flickering issues in complex scenes, we implement dynamic-aware regularization to maintain temporal consistency. This approach allows us to capture object motion and morphing over time while maintaining spatial coherence. To the best of our knowledge, ST-ObjGS is the first self-supervised approach using Space-time Gaussian surfels for consistent segmentation of dynamic 3D objects in real-world scenes. Extensive experiments on standard benchmarks including PKU-DyMVHumans, Plenoptic Video, Google Immersive, and CMU Panoptic datasets demonstrate that ST-ObjGS produces more precise object masks than its Gaussian-based counterparts and significantly outperforms supervised single-view baselines.