A technique for representing and learning smooth nonlinear manifolds is presented and applied to several lip reading tasks. Given a set of points drawn from a smooth manifold in an abstract feature space, the technique is capable of determining the structure of the surface and of finding the closest manifold point to a given query point. We use this technique to learn the "space of lips" in a visual speech recognition task. The learned manifold is used for tracking and extracting the lips, for interpolating between frames in an image sequence and for providing features for recognition. We describe a system based on hidden Markov models and this learned lip manifold that significantly improves the performance of acoustic speech recognizers in degraded environments. We also present preliminary results on a purely visual lip reader.<>


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Nonlinear manifold learning for visual speech recognition


    Contributors:


    Publication date :

    1995-01-01


    Size :

    757159 byte




    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English



    Nonlinear Manifold Learning for Visual Speech Recognition

    Bregler, C. / Omohundro, S. / IEEE Computer Society et al. | British Library Conference Proceedings | 1995


    Riemannian Manifold Learning for Nonlinear Dimensionality Reduction

    Lin, T. / Zha, H. / Lee, S. U. | British Library Conference Proceedings | 2006


    Learning the shape manifold to improve object recognition

    Chahooki, M. A. | British Library Online Contents | 2013


    Visual speech recognition by recurrent neural networks

    Rabi, G. / Si Wei Lu | British Library Online Contents | 1998


    Surge Monitoring Method Based on Nonlinear Manifold Learning

    Zhang, Y. / Xu, G. / Liang, L. | British Library Online Contents | 2009