Variational Autoencoders (VAE) and their variants have been widely used in a variety of applications, such as dialog generation, image generation and disentangled representation learning. However, the existing VAE models have some limitations in different applications. For example, a VAE easily suffers from KL vanishing in language modeling and low reconstruction quality for disentangling. To address these issues, we propose a novel controllable variational autoencoder framework, ControlVAE, that combines a controller, inspired by automatic control theory, with the basic VAE to improve the performance of resulting generative models. Specifically, we design a new non-linear PI controller, a variant of the proportional-integral-derivative (PID) control, to automatically tune the hyperparameter (weight) added in the VAE objective using the output KL-divergence as feedback during model training. The framework is evaluated using three applications; namely, language modeling, disentangled representation learning, and image generation. The results show that ControlVAE can achieve better disentangling and reconstruction quality than the existing methods. For language modelling, it not only averts the KL-vanishing, but also improves the diversity of generated text. Finally, we also demonstrate that ControlVAE improves the reconstruction quality of generated images compared to the original VAE.


    Zugriff

    Download


    Exportieren, teilen und zitieren



    Titel :

    ControlVAE: Controllable Variational Autoencoder


    Beteiligte:
    Shao, Huajie (Autor:in) / Yao, Shuochao (Autor:in) / Sun, Dachun (Autor:in) / Zhang, Aston (Autor:in) / Liu, Shengzhong (Autor:in) / Liu, Dongxin (Autor:in) / Wang, Jun (Autor:in) / Abdelzaher, Tarek (Autor:in)

    Erscheinungsdatum :

    2020


    Anmerkungen:

    accepted by ICML2020
    37th proceedings of ICML, 2020


    Medientyp :

    Preprint


    Format :

    Elektronische Ressource


    Sprache :

    Englisch




    Variational Autoencoder

    Okadome, Takeshi | Springer Verlag | 2025


    Variational Autoencoder

    Pinheiro Cinelli, Lucas / Araújo Marins, Matheus / Barros da Silva, Eduardo Antúnio et al. | Springer Verlag | 2021


    Autoencoding Variational Autoencoder

    Cemgil, A. Taylan / Ghaisas, Sumedh / Dvijotham, Krishnamurthy et al. | ArXiv | 2020

    Freier Zugriff

    Variational Lossy Autoencoder

    Chen, Xi / Kingma, Diederik P. / Salimans, Tim et al. | ArXiv | 2016

    Freier Zugriff

    Branched Variational Autoencoder Classifiers

    Salah, Ahmed / Yevick, David | ArXiv | 2024

    Freier Zugriff