Multimodal emotion recognition (MER), which relies on its role in processing and analysing comments posted on social media and identifying the corresponding target emotion states, has a very important position in education, social media and especially in the field of HCI. In order to solve the problem of emotion recognition for video, text and other forms of data in social media, we propose a Transformer with feature. The Transformer is able to take into account the combination of different feature relations and extract global keys, so it is introduced for long-range sequential context extraction, and the residual connections are placed after the Transformer to prevent information collapse. In addition, this paper uses a multi-headed attention mechanism to combine the extracted different feature laws, and finally the fused feature vectors are fed into the classifier to classify the sentiment and output it to achieve multimodal sentiment recognition in a multimodal context.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Multimodal Sentiment Recognition Based on Sentiment Fusion Transformer


    Contributors:
    Guan, Xinyu (author) / Wang, Yuan (author)


    Publication date :

    2023-10-11


    Size :

    2678047 byte




    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English



    A survey of multimodal sentiment analysis

    Soleymani, Mohammad / Garcia, David / Jou, Brendan et al. | British Library Online Contents | 2017


    Multimodal Sentiment Analysis Based on Image Captioning and Attention Mechanism

    Sun, Ye / Jin, Guozhe / Zhao, Yahui et al. | IEEE | 2023


    Multimodal Fusion-based Swin Transformer for Facial Recognition Micro-Expression Recognition

    Zhao, Xinhua / Lv, Yongjia / Huang, Zheng | British Library Conference Proceedings | 2022


    Guest editorial: Multimodal sentiment analysis and mining in the wild

    Soleymani, Mohammad / Schuller, Björn / Chang, Shih-Fu | British Library Online Contents | 2017


    Sensitive Information Recognition Based on Short Text Sentiment Analysis

    Li, Yang / Pan, Quan / Yang, Tao | British Library Online Contents | 2016