Enhancing Photorealism of Physically Accurate 3D Simulated Images Using GANs
2024
Reducing the realism gap between real and simulated sensor data remains a critical challenge in current research in modeling and simulation. Virtual Testbeds (VTBs) provide a safe, cost-effective environment for research, development, and testing but often fall short in the photorealism of the simulated camera sensors compared to real ones. To address this, we leveraged a state-of-the-art framework based on Generative Adversarial Networks (GANs) to enhance the photorealism of these VTBs. This paper introduces a generative Artificial Intelligence (AI) framework originally proposed to translate simulated image data from video games into photorealistic urban scenes. We adapt and extend this framework to different simulated data generated with a physically accurate 3D simulator. Various implementations with urban scenes were proposed and analyzed to assess their effectiveness in real-world scenarios. We evaluated our promising implementations using a real-time capable object detector to assess the impact of the enhancements and to identify persistent problems in enhancing the realism of simulated data.