Datasets are essential for training and testing vehicle perception algorithms. However, the collection and annotation of real-world images is time-consuming and expensive. Driving simulators offer a solution by automatically generating various driving scenarios with corresponding annotations, but the simulation-to-reality (Sim2Real) domain gap remains a challenge. While most of the Generative Artificial Intelligence (AI) follows the de facto Generative Adversarial Nets (GANs)-based methods, the recent emerging diffusion probabilistic models have not been fully explored in mitigating Sim2Real challenges for driving data synthesis. To explore the performance, this paper applied three different generative AI methods to leverage semantic label maps from a driving simulator as a bridge for the creation of realistic datasets. A comparative analysis of these methods is presented from the perspective of image quality and perception. New synthetic datasets, which include driving images and auto-generated high-quality annotations, are produced with low costs and high scene variability. The experimental results show that although GAN-based methods are adept at generating high-quality images when provided with manually annotated labels, ControlNet produces synthetic datasets with fewer artefacts and more structural fidelity when using simulator-generated labels. This suggests that the diffusion-based approach may provide improved stability and an alternative method for addressing Sim2Real challenges. These insights contribute to the intelligent vehicle community’s understanding of the potential for diffusion models to mitigate the Sim2Real gap.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Exploring Generative AI for Sim2Real in Driving Data Synthesis


    Contributors:


    Publication date :

    2024-06-02


    Size :

    2549138 byte





    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English



    Vision-based DRL Autonomous Driving Agent with Sim2Real Transfer

    Li, Dianzhao / Okhrin, Ostap | ArXiv | 2023

    Free access



    Segmented Encoding for Sim2Real of RL-based End-to-End Autonomous Driving

    Chung, Seung-Hwan / Kong, Seung-Hyun / Cho, Sangjae et al. | IEEE | 2022