Driving simulators play a large role in developing and testing new intelligent vehicle systems. The visual fidelity of the simulation is critical for building vision-based algorithms and conducting human driver experiments. Low visual fidelity breaks immersion for human-in-the-loop driving experiments. Conventional computer graphics pipelines use detailed 3D models, meshes, textures, and rendering engines to generate 2D images from 3D scenes. These processes are labor-intensive, and they do not generate photorealistic imagery. Here we introduce a hybrid generative neural graphics pipeline for improving the visual fidelity of driving simulations. Given a 3D scene, we partially-render only important objects of interest, such as vehicles, and use generative adversarial processes to synthesize the background and the rest of the image. To this end, we propose a novel image formation strategy to form 2D semantic images from 3D scenery consisting of simple object models without textures. These semantic images are then converted into photorealistic RGB images with a state-of-the-art Generative Adversarial Network (GAN) trained on real-world driving scenes. This replaces repetitiveness with randomly generated but photorealistic surfaces. Finally, the partially-rendered and GAN synthesized images are blended with a blending GAN. We show that the photorealism of images generated with the proposed method is more similar to real-world driving datasets such as Cityscapes and KITTI than conventional approaches. This comparison is made using semantic retention analysis and Frechet Inception Distance (FID) measurements.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    Photorealism in Driving Simulations: Blending Generative Adversarial Image Synthesis With Rendering


    Beteiligte:

    Erschienen in:

    Erscheinungsdatum :

    2022-12-01


    Format / Umfang :

    6604241 byte




    Medientyp :

    Aufsatz (Zeitschrift)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch



    GENERATIVE ADVERSARIAL NETWORK ENRICHED DRIVING SIMULATION

    SONG HAO / PENG JUN / DENG NENGXIU et al. | Europäisches Patentamt | 2020

    Freier Zugriff


    GENERATIVE ADVERSARIAL NETWORK ENRICHED DRIVING SIMULATION

    SONG HAO / PENG JUN / DENG NENGXIU et al. | Europäisches Patentamt | 2020

    Freier Zugriff

    Generative adversarial network enriched driving simulation

    SONG HAO / PENG JUN / DENG NENGXIU et al. | Europäisches Patentamt | 2022

    Freier Zugriff

    GENERATIVE ADVERSARIAL NETWORK ENRICHED DRIVING SIMULATION

    SONG HAO / PENG JUN / DENG NENGXIU et al. | Europäisches Patentamt | 2022

    Freier Zugriff