Emotion recognition is a domain of artificial intelligence that recognizes human emotions similar to the cognitive capabilities of Affective computing. It is an interdisciplinary field of computer science that is emerging as a new area for research. Recognizing emotions from different input sources using computational intelligence is an advanced application of artificial intelligence and has benefitted numerous applications in human-computer interaction, healthcare, and psychology. The proposed work extensively studies a multimodal approach to emotion recognition that takes input from audio and video modalities. The study critically analyses the performance and improvement of emotion recognition deep learning models and fusion techniques of audio-video input modalities. The proposed approach demonstrates the effectiveness of the multimodal approach and the challenges of multimodal fusion while building a model on RAVDESS, and IEMOCAP dataset. In essence, the research strives to contribute to the growing body of knowledge in the field of audio-based emotion recognition by providing a comprehensive analysis of feature extraction techniques and deep learning models. By understanding the strengths and limitations of each approach, researchers and practitioners can make informed decisions when developing systems that accurately capture and interpret emotions from audio and video modalities. Conclusively, the advancement has the potential to enhance human-computer interaction, improve mental health assessment, and enrich various other domains where emotion plays a pivotal role.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    Multimodal Approach: Emotion Recognition from Audio and Video Modality


    Beteiligte:
    Kumar, Himanshu (Autor:in) / Martin, A. (Autor:in)


    Erscheinungsdatum :

    22.11.2023


    Format / Umfang :

    447842 byte




    Medientyp :

    Aufsatz (Konferenz)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch



    Abnormal Emotion Recognition Based on Audio-Visual Modality Fusion

    Jiang, Yutong / Hirota, Kaoru / Dai, Yaping et al. | Springer Verlag | 2023


    Abnormal Emotion Recognition Based on Audio-Visual Modality Fusion

    Jiang, Yutong / Hirota, Kaoru / Dai, Yaping et al. | TIBKAT | 2023


    Audio-Visual Based Emotion Recognition -- A New Approach

    Song, M. / Bu, J. / Chen, C. et al. | British Library Conference Proceedings | 2004


    Audio-visual based emotion recognition - a new approach

    Mingli Song, / Jiajun Bu, / Chun Chen, et al. | IEEE | 2004


    Multimodal Emotion Recognition and Intention Understanding in Human-Robot Interaction

    Chen, Luefeng / Liu, Zhentao / Wu, Min et al. | Springer Verlag | 2021