We explain why directly changing the prior can be a surprisingly ineffective mechanism for incorporating inductive biases into VAEs, and introduce a simple and effective alternative approach: Intermediary Latent Space VAEs(InteL-VAEs). InteL-VAEs use an intermediary set of latent variables to control the stochasticity of the encoding process, before mapping these in turn to the latent representation using a parametric function that encapsulates our desired inductive bias(es). This allows us to impose properties like sparsity or clustering on learned representations, and incorporate human knowledge into the generative model. Whereas changing the prior only indirectly encourages behavior through regularizing the encoder, InteL-VAEs are able to directly enforce desired characteristics. Moreover, they bypass the computation and encoder design issues caused by non-Gaussian priors, while allowing for additional flexibility through training of the parametric mapping function. We show that these advantages, in turn, lead to both better generative models and better representations being learned.


    Zugriff

    Download


    Exportieren, teilen und zitieren



    Titel :

    On Incorporating Inductive Biases into VAEs


    Beteiligte:
    Miao, Ning (Autor:in) / Mathieu, Emile (Autor:in) / Siddharth, N. (Autor:in) / Teh, Yee Whye (Autor:in) / Rainforth, Tom (Autor:in)

    Erscheinungsdatum :

    2021


    Medientyp :

    Preprint


    Format :

    Elektronische Ressource


    Sprache :

    Englisch




    Disentangling Granularity: An Implicit Inductive Bias in Factorized VAEs

    Chen, Zihao / Xiang, Yu / Wang, Wenyong | ArXiv | 2025

    Freier Zugriff

    f-VAEs: Improve VAEs with Conditional Flows

    Su, Jianlin / Wu, Guang | ArXiv | 2018

    Freier Zugriff

    LEARNING HIERARCHICAL PRIORS IN VAES

    Klushyn, Alexej / Chen, Nutan / Kurle, Richard et al. | TIBKAT | 2020


    Relaxed-Responsibility Hierarchical Discrete VAEs

    Willetts, Matthew / Miscouridou, Xenia / Roberts, Stephen et al. | ArXiv | 2020

    Freier Zugriff

    Prior Learning in Introspective VAEs

    Athanasiadis, Ioannis / Lindsten, Fredrik / Felsberg, Michael | ArXiv | 2024

    Freier Zugriff