In this study, a vision based in-car entertainment user interface is presented. The user interface is designed using a hand posture and gesture recognition algorithm in deep learning framework. The hand posture recognition algorithm is formulated using the convolutional neural network to perform the fundamental tasks in the user interface. The hand gesture recognition algorithm is formulated using the long-term recurrent convolutional neural network to intuitively interact with the touchless automotive user interface in a detailed manner. In the recurrent deep learning framework, typically, the gesture frames are taken from a uniformly sampled image sequence. In this work, the recurrent structure is enhanced using a reduced number of input frames captured from the image sequence. The reduced input frames or key frames represent the action present in the video sequence. Sparse dictionary learning provide reliable key frame extraction from video sequences. However, sparse dictionary learning is computationally expensive, and are individually optimized for every video sequence. In this paper, we propose to approximate sparse dictionary learning using a non-linear regression framework. The multilayer perceptron is utilized to model the non-linear regression framework. The optimal neural network architecture is identified after a detailed evaluation. We evaluate the proposed recognition methods on public datasets. The proposed methods yield a recognition accuracy of 92% and 90% for pose and gestures, respectively. The combined hand posture and gesture recognition takes 82ms which is a reasonable for real time implementation.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Real-time hand posture and gesture-based touchless automotive user interface using deep learning


    Contributors:
    John, V. (author) / Umetsu, M. (author) / Boyali, A. (author) / Mita, S. (author) / Imanishi, M. (author) / Sanma, N. (author) / Shibata, S. (author)


    Publication date :

    2017-06-01


    Size :

    1231404 byte




    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English



    Touchless Elevator User Interface

    YERAZUNIS WILLIAM / MAHMUD ALEEF / LEE BRIAN | European Patent Office | 2022

    Free access

    Real-Time Hand Gesture Recognition for Robot Hand Interface

    Lv, Xiaomeng / Xu, Yulin / Wang, Ming | Springer Verlag | 2014


    TOUCHLESS GESTURE RECOGNITION FOR ELEVATOR SERVICE

    SCOVILLE BRADLEY ARMAND / SIMCIK PAUL A / PETERSON ERIC C | European Patent Office | 2017

    Free access

    Real-Time Hand Gesture Recognition for Robot Hand Interface

    Lv, X. / Xu, Y. / Wang, M. | British Library Conference Proceedings | 2014


    Touchless gesture recognition for elevator service

    SCOVILLE BRADLEY ARMAND / SIMCIK PAUL A / PETERSON ERIC C | European Patent Office | 2018

    Free access