Multimodal Variational Autoencoders (VAEs) represent a promising group of generative models that facilitate the construction of a tractable posterior within the latent space given multiple modalities. Previous studies have shown that as the number of modalities increases, the generative quality of each modality declines. In this study, we explore an alternative approach to enhance the generative performance of multimodal VAEs by jointly modeling the latent space of independently trained unimodal VAEs using score-based models (SBMs). The role of the SBM is to enforce multimodal coherence by learning the correlation among the latent variables. Consequently, our model combines a better generative quality of unimodal VAEs with coherent integration across different modalities using the latent score-based model. In addition, our approach provides the best unconditional coherence.


    Zugriff

    Download


    Exportieren, teilen und zitieren



    Titel :

    Score-Based Multimodal Autoencoder


    Beteiligte:

    Erscheinungsdatum :

    2023


    Medientyp :

    Preprint


    Format :

    Elektronische Ressource


    Sprache :

    Englisch





    Vehicular Multimodal Motion Forecasting via Conditional Score-based Modeling

    Wang, Zhangyun / Ning, Nianwen / Tian, Shihan et al. | IEEE | 2023



    Variational Autoencoder

    Okadome, Takeshi | Springer Verlag | 2025


    Variational Autoencoder

    Pinheiro Cinelli, Lucas / Araújo Marins, Matheus / Barros da Silva, Eduardo Antúnio et al. | Springer Verlag | 2021