We present an approach to detecting and recognizing spoken isolated phrases based solely on visual input. We adopt an architecture that first employs discriminative detection of visual speech and articulate features, and then performs recognition using a model that accounts for the loose synchronization of the feature streams. Discriminative classifiers detect the subclass of lip appearance corresponding to the presence of speech, and further decompose it into features corresponding to the physical components of articulate production. These components often evolve in a semi-independent fashion, and conventional viseme-based approaches to recognition fail to capture the resulting co-articulation effects. We present a novel dynamic Bayesian network with a multi-stream structure and observations consisting of articulate feature classifier scores, which can model varying degrees of co-articulation in a principled way. We evaluate our visual-only recognition system on a command utterance task. We show comparative results on lip detection and speech/non-speech classification, as well as recognition performance against several baseline systems.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    Visual speech recognition with loosely synchronized feature streams


    Beteiligte:
    Saenko, K. (Autor:in) / Livescu, K. (Autor:in) / Siracusa, M. (Autor:in) / Wilson, K. (Autor:in) / Glass, J. (Autor:in) / Darrell, T. (Autor:in)


    Erscheinungsdatum :

    2005-01-01


    Format / Umfang :

    198526 byte





    Medientyp :

    Aufsatz (Konferenz)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch



    Visual Speech Recognition with Loosely Synchronized Feature Streams

    Saenko, K. / Livescu, K. / Siracusa, M. et al. | British Library Conference Proceedings | 2005


    Dynamic Depth Recovery from Multiple Synchronized Video Streams

    Tao, H. / Sawhney, H. S. / Kumar, R. et al. | British Library Conference Proceedings | 2001


    Dynamic depth recovery from multiple synchronized video streams

    Hai Tao, / Sawhney, H.S. / Kumar, R. | IEEE | 2001


    Wipes, Cotton, Loosely Woven

    SAE Technical Papers | 2016


    WIPES, COTTON, LOOSELY WOVEN

    SAE Technical Papers | 2011