The posterior collapse phenomenon in variational autoencoder (VAE), where the variational posterior distribution closely matches the prior distribution, can hinder the quality of the learned latent variables. As a consequence of posterior collapse, the latent variables extracted by the encoder in VAE preserve less information from the input data and thus fail to produce meaningful representations as input to the reconstruction process in the decoder. While this phenomenon has been an actively addressed topic related to VAE performance, the theory for posterior collapse remains underdeveloped, especially beyond the standard VAE. In this work, we advance the theoretical understanding of posterior collapse to two important and prevalent yet less studied classes of VAE: conditional VAE and hierarchical VAE. Specifically, via a non-trivial theoretical analysis of linear conditional VAE and hierarchical VAE with two levels of latent, we prove that the cause of posterior collapses in these models includes the correlation between the input and output of the conditional VAE and the effect of learnable encoder variance in the hierarchical VAE. We empirically validate our theoretical findings for linear conditional and hierarchical VAE and demonstrate that these results are also predictive for non-linear cases with extensive experiments.


    Zugriff

    Download


    Exportieren, teilen und zitieren



    Titel :

    Beyond Vanilla Variational Autoencoders: Detecting Posterior Collapse in Conditional and Hierarchical Variational Autoencoders


    Beteiligte:
    Dang, Hien (Autor:in) / Tran, Tho (Autor:in) / Nguyen, Tan (Autor:in) / Ho, Nhat (Autor:in)

    Erscheinungsdatum :

    2023


    Anmerkungen:

    Accepted (Poster) at the Twelfth International Conference on Learning Representations


    Medientyp :

    Preprint


    Format :

    Elektronische Ressource


    Sprache :

    Englisch




    Variational Autoencoders

    Ghojogh, Benyamin / Crowley, Mark / Karray, Fakhri et al. | Springer Verlag | 2022


    Learning Conditional Variational Autoencoders with Missing Covariates

    Ramchandran, Siddharth / Tikhonov, Gleb / Lönnroth, Otto et al. | ArXiv | 2022

    Freier Zugriff

    Deep Tracking Portfolios Using Autoencoders and Variational Autoencoders

    Urrego, Daniel Aragón / Nieto, Oscar Eduardo Reyes / Quimbayo, Carlos Andrés Zapata | Springer Verlag | 2024


    Certifiably Robust Variational Autoencoders

    Barrett, Ben / Camuto, Alexander / Willetts, Matthew et al. | ArXiv | 2021

    Freier Zugriff

    Mixed-curvature Variational Autoencoders

    Skopek, Ondrej / Ganea, Octavian-Eugen / Bécigneul, Gary | ArXiv | 2019

    Freier Zugriff