```html

From Simulation to Reality: Dynamic Digital Twins Bridge the Gap

Robotics has made tremendous progress in recent years, particularly in the field of machine learning. Complex manipulation tasks that previously required human precision can now be performed autonomously by robots. A central aspect of this development is so-called Behavior Cloning, where robots learn by observing and imitating human demonstrations. Despite this progress, the reliable evaluation of training progress remains a challenge. Common evaluation metrics for Behavior Cloning often correlate poorly with actual success in real-world application scenarios.

Traditionally, researchers resort to the success rate in real-world tests to evaluate the performance of robot strategies. However, this method is time-consuming, cost-intensive, and makes it difficult to identify optimal strategies and to detect overfitting or underfitting. A promising approach to bridge this gap between simulation and reality is the use of digital twins.

A recent research project, "Real-is-Sim", presents a novel Behavior Cloning framework that integrates a dynamic digital twin. This digital twin, based on Embodied Gaussians, accompanies the entire development process of the robot strategy: from data acquisition and training to application. By continuously synchronizing the simulated world with physical reality, demonstrations can be collected in the real world, while the states are extracted from the simulation.

The simulator offers flexible representation possibilities for states. For example, image data can be rendered from arbitrary viewpoints, or detailed information about the objects in the scenario can be extracted. The training of the strategies can be performed directly in the simulator, enabling offline evaluation and high parallelization. In the application phase, the real robot controls the joints of the simulated robot, thereby decoupling the strategy execution from the real hardware and minimizing challenges in domain transfer.

The validation of "Real-is-Sim" was carried out using the manipulation task "PushT". The results show a strong correlation between the success rates achieved in the simulator and in the real world. This approach promises a more efficient and cost-effective development of robot strategies by closing the gap between simulation and reality and enabling a more realistic evaluation of training performance.

The integration of dynamic digital twins into the development process of robot strategies represents an important step towards overcoming the challenges of sim-to-real transfer. Through continuous synchronization between simulation and reality, training data can be collected more efficiently, strategies can be optimized, and performance can be evaluated more reliably. This opens up new possibilities for the development of robust and powerful robot systems in a variety of application areas.

Bibliography: Abou-Chakra, J., Sun, L., Rana, K., May, B., Schmeckpeper, K., Minniti, M. V., & Herlant, L. (2025). Real-is-Sim: Bridging the Sim-to-Real Gap with a Dynamic Digital Twin for Real-World Robot Policy Evaluation. arXiv preprint arXiv:2504.03597. Autodesk Research. Sim-to-Real Gap. Fang, K., Zhou, Y., Zhao, D., & Bian, G. B. (2025). Sim-to-real: Adaptive domain randomization via sample-based discrepancies. Journal of Manufacturing Systems, 71, 261-272. James, S., Wohlhart, P., Kalweit, M., Gesenhues, U., & Asfour, T. (2024). Sim-to-real transfer for manipulation using self-supervised object keypoint detection. arXiv preprint arXiv:2403.03949. Li, Y., Chen, Y., & Zhao, H. (2022). A digital twin-based sim-to-real transfer for deep reinforcement learning-enabled industrial robot grasping. Robotics and Computer-Integrated Manufacturing, 76, 102317. Raileanu, R., & Fergus, R. (2018). Decoupling feature extraction from policy learning: assessing benefits of state representation learning in goal based robotics. Song, H., Sachar, S., Jayaraman, D., & Saxena, A. (2022). Sim-to-Real Transfer in Robotics: Addressing the Gap between Simulation and Real-World Performance. arXiv preprint arXiv:2208.01677. Zhao, W., Queralta, J. P., Westerlund, T., & González, D. (2022). Sim-to-real transfer learning of pick and place tasks for industrial robots. Assembly Automation, 42(3), 424-436. ```