CityDreamer4D: A New Approach to Generative Modeling of 4D Cities

The generation of 3D scenes has gained significant importance and achieved remarkable progress in recent years. However, the creation of dynamic, i.e., 4D cities, presents an even greater challenge. This is partly due to the complexity and visual diversity of urban structures, such as buildings and vehicles, and partly due to the increased human sensitivity to distortions in urban environments. A new research approach called CityDreamer4D addresses precisely this challenge.

Compositional Generation for Unlimited 4D Cities

CityDreamer4D is a compositional generative model specifically designed for generating unlimited 4D cities. The approach is based on two central insights: First, the generation of 4D cities should separate dynamic objects (e.g., vehicles) from static scenes (e.g., buildings and roads). Second, all objects in the 4D scene should be composed from different types of neural fields for buildings, vehicles, and background elements.

Core Components and Functionality

To implement these principles, CityDreamer4D uses a "Traffic Scenario Generator" and an "Unbounded Layout Generator." The Traffic Scenario Generator creates dynamic traffic scenarios, while the Unbounded Layout Generator creates static city layouts using a compact BEV (Bird's-Eye-View) representation. The objects in the 4D cities are generated by combining material- and instance-oriented neural fields for background, buildings, and vehicles. For scene parameterization, the neural fields use customized generative hash grids and periodic positional encodings tailored to the specific properties of background and instances.

Comprehensive Datasets for Realistic City Generation

Another important aspect of CityDreamer4D is the use of comprehensive datasets for city generation. These include OSM (OpenStreetMap), Google Earth, and CityTopia. OSM provides diverse real-world city layouts, while Google Earth and CityTopia provide large amounts of high-resolution city imagery with 3D instance annotations. These datasets enable CityDreamer4D to generate realistic and detailed 4D cities.

Diverse Application Possibilities

Thanks to its compositional design, CityDreamer4D supports several downstream applications, including:

Instance Editing: Individual objects within the generated city can be specifically modified or removed.

City Stylization: The style of the generated city can be adjusted, e.g., by applying different architectural styles.

Urban Simulation: CityDreamer4D can be used for simulations in urban spaces, e.g., for traffic simulation or for planning urban development projects.

CityDreamer4D and Mindverse: Synergies for the Future

The development of CityDreamer4D represents a significant advance in the field of generative AI. For companies like Mindverse, which specialize in AI-powered content creation, this opens up new possibilities. The technology could, for example, be integrated into Mindverse's content platform to enable customers to create realistic and dynamic 4D city models. The development of customized AI solutions, such as chatbots or knowledge databases, could also benefit from the advances in the field of 4D city generation.

Bibliographie: - https://www.infinitescript.com/project/city-dreamer/ - https://www.infinitescript.com/project/city-dreamer-4d/ - https://github.com/hzxie - https://www.chatpaper.com/chatpaper/fr?id=4&date=1736956800&page=1