```html

Reconstructing Dynamic 3D Scenes from Monocular Videos: A New Approach through Video Inpainting

The reconstruction of dynamic 3D scenes, also known as 4D reconstruction, from everyday smartphone videos is a complex and promising research area. The challenge lies in accurately capturing the spatial depth and temporal evolution of the scene from a single perspective offered by a monocular video. A new approach called Vivid4D promises improvements here by expanding the viewing angles using video inpainting.

The Vivid4D Approach: Expanding Perspectives

Vivid4D pursues an innovative approach by using the existing views from the monocular video to generate additional, synthetic views. This expansion of perspectives allows for a more comprehensive representation of the scene and improves the quality of the 4D reconstruction. In contrast to previous methods, which rely either exclusively on geometric priors or on generative models, Vivid4D combines both.

Specifically, the expansion of views is formulated as a video inpainting task. Using depth information estimated from the monocular video, the existing image content is projected into new viewpoints. This creates gaps and occlusions that must be filled by the inpainting model. The model is trained on freely available, unposed videos to which synthetically generated masks are added to simulate the gaps created by the projection.

Training and Optimization of the Inpainting Model

The training of the video inpainting model is crucial for the success of Vivid4D. By using synthetic masks that simulate the occlusions typically occurring during projection into new viewpoints, the model learns to complete missing image areas spatially and temporally consistently. This leads to an improved quality of the synthesized views and ultimately to a more precise 4D reconstruction.

Handling Inaccuracies in Depth Estimations

The accuracy of the depth information extracted from the monocular video plays a crucial role in the quality of the synthesized views. To minimize inaccuracies in the depth estimations, Vivid4D uses an iterative strategy for view expansion. Additionally, a robust loss function is employed, which reduces the impact of errors in the depth estimations on the overall result.

Results and Outlook

Experimental results show that Vivid4D significantly improves 4D reconstruction from monocular videos. The combination of geometric priors and generative models in conjunction with the iterative view expansion and the robust loss function leads to higher accuracy and detail fidelity of the reconstructed scenes. Future research could focus on further improving depth estimation and optimizing the inpainting model to further enhance the quality of 4D reconstructions.

The development of Vivid4D underscores the potential of AI-based methods for solving complex tasks in computer vision. The ability to reconstruct dynamic 3D scenes from everyday videos opens up diverse application possibilities in areas such as virtual reality, augmented reality, robotics, and autonomous driving.

Bibliographie: https://arxiv.org/abs/2504.11092 https://arxiv.org/html/2504.11092v1 https://paperreading.club/page?id=299841 https://x.com/zhenjun_zhao/status/1912457772760789271 https://huggingface.co/papers/2407.13764 https://proceedings.neurips.cc/paper_files/paper/2024/file/ed3c686f9cda57e56cc859402c775414-Paper-Conference.pdf http://140.143.194.41/category?cate=Reconstruction https://www.researchgate.net/publication/373136988_Unbiased_4D_Monocular_4D_Reconstruction_with_a_Neural_Deformation_Model https://4dqv.mpi-inf.mpg.de/Ub4D/ ```