We propose a new approach for combining acoustic and visual measurements to aid in recognizing lip shapes of a person speaking. Our method relies on computing the maximum likelihoods of (a) HMM used to model phonemes from the acoustic signal, and (b) HMM used to model visual features motions from video. One significant addition is the dynamic analysis with features selected by AdaBoost, on the basis of their discriminant ability. This form of integration, leading to boosted HMM, permits AdaBoost to find the best features first, and then uses HMM to exploit dynamic information inherent in the signal.
Boosted audio-visual HMM for speech reading
2003-01-01
289856 byte
Aufsatz (Konferenz)
Elektronische Ressource
Englisch
Boosted Audio-Visual HMM for Speech Reading
British Library Conference Proceedings | 2003
|Asymmetrically Boosted HMM for Speech Reading
British Library Conference Proceedings | 2004
|Asymmetrically boosted HMM for speech reading
IEEE | 2004
|Audio-Visual Speech Recognition using SciPy
TIBKAT | 2010
|Measuring Audio and Visual Speech Synchrony: Methods and Applications
British Library Conference Proceedings | 2006
|