Efficiently Reconstructing Dynamic 3D Scenes: MoDec-GS Reduces Memory Requirements and Improves Rendering

3D Gaussian Splatting (3DGS) has established itself as a promising method for scene representation and neural rendering. The technology enables impressive rendering quality and speed, especially in the reconstruction of static scenes. However, adapting it for dynamic scenes presents a challenge due to increased memory requirements and the difficulty of representing complex motions. A new approach called MoDec-GS promises a solution.

MoDec-GS (Motion Decomposition Gaussian Splatting) is a memory-efficient framework for reconstructing views of dynamic scenes with complex motions. At its core is the Global-to-Local Motion Decomposition (GLMD), which captures dynamic movements hierarchically. It uses global and local Canonical Scaffolds (CS), an extension of the static scaffold concept for dynamic video reconstruction.

Global CS represent the entire scene across all frames. The Global Anchor Deformation (GAD) enables efficient representation of global motions by directly modifying the implicit scaffold attributes: anchor position, offset, and local context features. Local CS, on the other hand, focus on individual temporal segments. Here, the Local Gaussian Deformation (LGD) refines local motions through explicit deformation of the reconstructed 3D Gaussians.

Another important component of MoDec-GS is the Temporal Interval Adjustment (TIA). This method automatically controls the temporal coverage of each local CS during training. This allows MoDec-GS to find optimal interval assignments based on the number of temporal segments. This leads to an efficient adaptation to the degree of motion within the scene.

Evaluations show that MoDec-GS achieves an average reduction in model size of 70% compared to state-of-the-art methods for dynamic 3D Gaussians from real videos. At the same time, rendering quality is maintained or even improved. This opens up new possibilities for applications that require both high visual quality and real-time performance, such as augmented and virtual reality, gaming, or interactive 3D models.

The combination of GLMD and TIA allows MoDec-GS to capture complex motions efficiently and accurately. The hierarchical decomposition of motion into global and local components significantly reduces memory requirements, while the automatic adjustment of temporal intervals optimizes rendering quality. MoDec-GS thus represents a significant advance in the field of dynamic 3D Gaussian Splatting.

Bibliography

Kwak, S., Kim, J., Jeong, J. Y., Cheong, W.-S., Oh, J., & Kim, M. (2025). MoDec-GS: Global-to-Local Motion Decomposition and Temporal Interval Adjustment for Compact Dynamic 3D Gaussian Splatting. arXiv preprint arXiv:2501.03714.

Javed, S., Khan, A. J., Dumery, C., Zhao, C., & Salzmann, M. (2024). Temporally Compressed 3D Gaussian Splatting for Dynamic Scenes. arXiv preprint arXiv:2412.05700.

Xu, W., Weng, W., Zhang, Y., Xu, R., & Xiong, Z. (2024). Event-boosted Deformable 3D Gaussians for Fast Dynamic Scene Reconstruction. arXiv preprint arXiv:2411.16180v1.

Fischer, T., Kulhanek, J., Bulò, S. R., Porzi, L., Pollefeys, M., & Kontschieder, P. (2024). Dynamic 3D Gaussian Fields for Urban Areas. Advances in Neural Information Processing Systems, 37.

https://kaist-viclab.github.io/MoDecGS-site/

https://paperreading.club/page?id=277144

https://arxiv.org/abs/2412.05700

https://github.com/Awesome3DGS/3D-Gaussian-Splatting-Papers

https://arxiv.org/html/2411.16180v1

https://github.com/Lee-JaeWon/2024-Arxiv-Paper-List-Gaussian-Splatting

https://cvpr.thecvf.com/virtual/2024/session/32086

https://eccv.ecva.net/virtual/2024/poster/1662

https://openreview.net/forum?id=xZxXNhndXU&referrer=%5Bthe%20profile%20of%20Marc%20Pollefeys%5D(%2Fprofile%3Fid%3D~Marc_Pollefeys2)

https://paperreading.club/category?cate=Reconstruction