Efficient Synthesis of Novel Views of Large-Scale Scenes with BlockGaussian

The realistic rendering of 3D scenes from arbitrary viewpoints, known as Novel View Synthesis, is a central research area in computer graphics and computer vision. Recent advancements in 3D Gaussian Splatting (3DGS) have achieved impressive results, but challenges remain, particularly in the reconstruction of large-scale scenes, regarding scene partitioning, optimization, and merging of individual parts. A new approach called BlockGaussian promises a remedy.

BlockGaussian: An Overview

BlockGaussian is a novel framework that combines a content-aware scene partitioning strategy and view-dependent optimized block processing to enable efficient and high-quality reconstruction of large-scale scenes. The approach considers the varying complexity of content in different areas of the scene and distributes the computational load during partitioning to ensure efficient reconstruction.

Content-Aware Scene Partitioning

Instead of dividing the scene into equally sized blocks, BlockGaussian analyzes the content and partitions the scene so that areas with high detail are subdivided more finely than areas with low complexity. This allows for optimal utilization of computational resources and leads to a more even distribution of the computational load.

View-Dependent Block Optimization

To address the issue of supervision inconsistency during independent block optimization, BlockGaussian introduces auxiliary points. These auxiliary points serve to adjust the ground-truth supervision during the optimization of individual blocks, thereby improving the quality of the reconstruction. By considering the visibility of each block, the accuracy of the reconstruction is further enhanced.

Mitigating Artifacts through Pseudo-Visibility Geometry

Another problem when merging individual blocks is so-called "Airspace Floaters," artifacts that can arise from inaccuracies in the reconstruction. BlockGaussian uses a pseudo-visibility geometry constraint to minimize these artifacts and improve the quality of the rendered images.

Experimental Results

Extensive experiments with large-scale scenes show that BlockGaussian achieves state-of-the-art results in terms of both reconstruction efficiency and rendering quality. Compared to existing methods, a five-fold acceleration of optimization and an average PSNR improvement of 1.21 dB were achieved on various benchmarks. Particularly noteworthy is that BlockGaussian significantly reduces the computational requirements and enables the reconstruction of large-scale scenes on a single device with 24 GB VRAM.

Conclusion

BlockGaussian represents a significant advance in the field of Novel View Synthesis. By combining content-aware scene partitioning, view-dependent block optimization, and the use of pseudo-visibility geometry constraints, BlockGaussian enables efficient and high-quality reconstruction of large-scale scenes. The reduction in computational requirements opens up new possibilities for the application of 3DGS in various fields, from virtual reality and augmented reality to architecture and urban planning.

Bibliographie: https://arxiv.org/html/2504.09048v1 https://chatpaper.com/chatpaper/zh-CN/paper/129252 https://github.com/SunshineWYC/BlockGaussian https://x.com/janusch_patas/status/1912029513606987808 https://twitter.com/zhenjun_zhao/status/1912159613304070256 https://www.ecva.net/papers/eccv_2024/papers_ECCV/papers/02472.pdf https://twitter.com/janusch_patas/status/1912029526084952391 https://radiancefields.com/ https://eccv.ecva.net/virtual/2024/poster/178