Synthesizing a realistic image from textual description is a major challenge in computer vision. Current text to image synthesis approaches falls short of producing a high-resolution image that represent a text descriptor. Most existing studies rely either on Generative Adversarial Networks (GANs) or Variational Auto Encoders (VAEs). GANs has the capability to produce sharper images but lacks the diversity of outputs, whereas VAEs are good at producing a diverse range of outputs, but the images generated are often blurred. Taking into account the relative advantages of both GANs and VAEs, we proposed a new stacked Conditional VAE (CVAE) and Conditional GAN (CGAN) network architecture for synthesizing images conditioned on a text description. This study uses Conditional VAEs as an initial generator to produce a high-level sketch of the text descriptor. This high-level sketch output from first stage and a text descriptor is used as an input to the conditional GAN network. The second stage GAN produces a 256 × 256 high resolution image. The proposed architecture benefits from a conditioning augmentation and a residual block on the Conditional GAN network to achieve the results. Multiple experiments were conducted using CUB and Oxford-102 dataset and the result of the proposed approach is compared against state-of-the-art techniques such as StackGAN. The experiments illustrate that the proposed method generates a high-resolution image conditioned on text descriptions and yield competitive results based on Inception and Fréchet Inception Score using both datasets.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    Text to Image Synthesis Using Stacked Conditional Variational Autoencoders and Conditional Generative Adversarial Networks


    Weitere Titelangaben:

    Lect. Notes in Networks, Syst.


    Beteiligte:
    Arai, Kohei (Herausgeber:in) / Tibebu, Haileleol (Autor:in) / Malik, Aadin (Autor:in) / De Silva, Varuna (Autor:in)

    Kongress:

    Science and Information Conference ; 2022 July 14, 2022 - July 15, 2022


    Erschienen in:

    Intelligent Computing ; Kapitel : 38 ; 560-580


    Erscheinungsdatum :

    07.07.2022


    Format / Umfang :

    21 pages





    Medientyp :

    Aufsatz/Kapitel (Buch)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch




    Learning Conditional Variational Autoencoders with Missing Covariates

    Ramchandran, Siddharth / Tikhonov, Gleb / Lönnroth, Otto et al. | ArXiv | 2022

    Freier Zugriff


    Multiview Scene Image Inpainting Based on Conditional Generative Adversarial Networks

    Yuan, Zefeng / Li, Hengyu / Liu, Jingyi et al. | IEEE | 2020


    REALISTIC ULTRASONIC ENVIRONMENT SIMULATION USING CONDITIONAL GENERATIVE ADVERSARIAL NETWORKS

    Pöpperli, Maximilian / Gulagundi, Raghavendra / Yogamani, Senthil et al. | British Library Conference Proceedings | 2019


    Realistic Ultrasonic Environment Simulation Using Conditional Generative Adversarial Networks

    Popperli, Maximilian / Gulagundi, Raghavendra / Yogamani, Senthil et al. | IEEE | 2019