Multimodal Variational Autoencoders (VAEs) represent a promising group of generative models that facilitate the construction of a tractable posterior within the latent space given multiple modalities. Previous studies have shown that as the number of modalities increases, the generative quality of each modality declines. In this study, we explore an alternative approach to enhance the generative performance of multimodal VAEs by jointly modeling the latent space of independently trained unimodal VAEs using score-based models (SBMs). The role of the SBM is to enforce multimodal coherence by learning the correlation among the latent variables. Consequently, our model combines a better generative quality of unimodal VAEs with coherent integration across different modalities using the latent score-based model. In addition, our approach provides the best unconditional coherence.


    Access

    Download


    Export, share and cite



    Title :

    Score-Based Multimodal Autoencoder


    Contributors:

    Publication date :

    2023


    Type of media :

    Preprint


    Type of material :

    Electronic Resource


    Language :

    English





    Vehicular Multimodal Motion Forecasting via Conditional Score-based Modeling

    Wang, Zhangyun / Ning, Nianwen / Tian, Shihan et al. | IEEE | 2023



    Variational Autoencoder

    Okadome, Takeshi | Springer Verlag | 2025


    Autoencoding Variational Autoencoder

    Cemgil, A. Taylan / Ghaisas, Sumedh / Dvijotham, Krishnamurthy et al. | ArXiv | 2020

    Free access