A technique for representing and learning smooth nonlinear manifolds is presented and applied to several lip reading tasks. Given a set of points drawn from a smooth manifold in an abstract feature space, the technique is capable of determining the structure of the surface and of finding the closest manifold point to a given query point. We use this technique to learn the "space of lips" in a visual speech recognition task. The learned manifold is used for tracking and extracting the lips, for interpolating between frames in an image sequence and for providing features for recognition. We describe a system based on hidden Markov models and this learned lip manifold that significantly improves the performance of acoustic speech recognizers in degraded environments. We also present preliminary results on a purely visual lip reader.<>
Nonlinear manifold learning for visual speech recognition
1995-01-01
757159 byte
Conference paper
Electronic Resource
English
Nonlinear Manifold Learning for Visual Speech Recognition
British Library Conference Proceedings | 1995
|Riemannian Manifold Learning for Nonlinear Dimensionality Reduction
British Library Conference Proceedings | 2006
|Learning the shape manifold to improve object recognition
British Library Online Contents | 2013
|Visual speech recognition by recurrent neural networks
British Library Online Contents | 1998
|Surge Monitoring Method Based on Nonlinear Manifold Learning
British Library Online Contents | 2009
|