In indoor places, such as homes or offices, when abnormal events occur, the behavior and voice of individuals or groups will display abnormal signals. These signals can be both visual and auditory, and they interact and complement each other to jointly create a sense of emotional atmosphere within the scene. In order to achieve effective and accurate perception and response of abnormal emotion during the interaction in smart home, a model of abnormal emotion recognition based on audio-visual modality fusion is proposed. Human skeleton motion data and audio data are utilized to construct separate deep learning networks for action recognition and speech emotion recognition. The accuracy rate achieved on the G3D dataset is 100% and the accuracy rate achieved on the CASIA corpus is 90.83%. For decision-level multimodal fusion, the predicted results of actions and speech emotions are mapped to the “abnormal” axis through fuzzification and weighted average methods. In this process, considerations are taken into account for the varying contributions of different speech emotions and behaviors to the abnormal emotion, as well as the recognition recall rates of the unimodal emotion models. Then the two modalities are allowed to mutually modify each other and achieve quantitative analysis of abnormal emotion through weighted additive fusion.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    Abnormal Emotion Recognition Based on Audio-Visual Modality Fusion


    Weitere Titelangaben:

    Lect.Notes Computer


    Beteiligte:
    Yang, Huayong (Herausgeber:in) / Liu, Honghai (Herausgeber:in) / Zou, Jun (Herausgeber:in) / Yin, Zhouping (Herausgeber:in) / Liu, Lianqing (Herausgeber:in) / Yang, Geng (Herausgeber:in) / Ouyang, Xiaoping (Herausgeber:in) / Wang, Zhiyong (Herausgeber:in) / Jiang, Yutong (Autor:in) / Hirota, Kaoru (Autor:in)

    Kongress:

    International Conference on Intelligent Robotics and Applications ; 2023 ; Hangzhou, China July 05, 2023 - July 07, 2023



    Erscheinungsdatum :

    21.10.2023


    Format / Umfang :

    12 pages




    Medientyp :

    Aufsatz/Kapitel (Buch)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch




    Abnormal Emotion Recognition Based on Audio-Visual Modality Fusion

    Jiang, Yutong / Hirota, Kaoru / Dai, Yaping et al. | TIBKAT | 2023



    Audio-Visual Based Emotion Recognition -- A New Approach

    Song, M. / Bu, J. / Chen, C. et al. | British Library Conference Proceedings | 2004


    Audio-visual based emotion recognition - a new approach

    Mingli Song, / Jiajun Bu, / Chun Chen, et al. | IEEE | 2004


    CMM: LiDAR-Visual Fusion with Cross-Modality Module for Large-Scale Place Recognition

    Li, Bin / Lu, Fan / Chen, Guang et al. | SAE Technical Papers | 2023