Advances in computer processing power and emerging algorithms are allowing new ways of envisioning human computer interaction. This paper focuses on the development of a computing algorithm that uses audio and visual sensors to detect and track a user's affective state to aid computer decision making. Using our multi-stream fused hidden Markov model (MFHMM), we analyzed coupled audio and visual streams to detect 11 cognitive/emotive states. The MFHMM allows the building of an optimal connection among multiple streams according to the maximum entropy principle and the maximum mutual information criterion. Person-independent experimental results from 20 subjects in 660 sequences show that the MFHMM approach performs with an accuracy of 80.61% which outperforms face-only HMM, pitch-only HMM, energy-only HMM, and independent HMM fusion.
Audio-visual affect recognition through multi-stream fused HMM for HCI
2005-01-01
166253 byte
Aufsatz (Konferenz)
Elektronische Ressource
Englisch
Audio-Visual Speech Recognition using SciPy
TIBKAT | 2010
|Audio-visual based emotion recognition - a new approach
IEEE | 2004
|Audio-Visual Based Emotion Recognition -- A New Approach
British Library Conference Proceedings | 2004
|