A large number of demands for space on-orbit services to ensure the on-orbit system completes its specified tasks are foreseeable, and the efficiency and the security are the most significant factors when we carry out an on-orbit mission. And it can improve human-computer interaction efficiency in operations with proper gesture recognition solutions. In actual situations, the operations are complex and changeable, so the gestures used in interaction are also difficult to predict in advance due to the compounding of multiple consecutive gestures. To recognize such gestures based on computer vision (CV) requires complex models trained by a large amount of datasets, it is often unable to obtain enough gesture samples for training a complex model in real tasks, and the cost of labeling the collected gesture samples is quite expensive. Aiming at the problems mentioned above, we propose a few-shot continuous gesture recognition scheme based on RGB video. The scheme uses Mediapipe to detect the key points of each frame in the video stream, decomposes the basic components of gesture features based on certain human palm structure, and then extracts and combines the above basic gesture features by a lightweight autoencoder network. Our scheme can achieve 89.73% recognition accuracy on the 5-way 1-shot gesture recognition task which randomly selected 142 gesture instances of 5 categories from the RWTH German fingerspelling dataset.


    Access

    Download


    Export, share and cite



    Title :

    Continuous Gesture Sequences Recognition Based on Few-Shot Learning


    Contributors:
    Zhe Liu (author) / Cao Pan (author) / Hongyuan Wang (author)


    Publication date :

    2022




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    Unknown




    Gesture Recognition Using Stereo Image Sequences

    Onoguchi, K. / Sato, M. | British Library Online Contents | 2007


    Gesture sequence recognition with one shot learned CRF/HMM hybrid model

    Belgacem, Selma / Chatelain, Clément / Paquet, Thierry | British Library Online Contents | 2017


    Improving continuous gesture recognition with spoken prosody

    Kettebekov, S. / Yeasin, M. / Sharma, R. | IEEE | 2003


    Position-Free Hand Gesture Recognition Using Single Shot MultiBox Detector Based Neural Network

    Tang, Jingwei / Yao, Xingtian / Kang, Xin et al. | British Library Conference Proceedings | 2019


    Improving Continuous Gesture Recognition with Spoken Prosody

    Kettebekov, S. / Yeasin, M. / Sharma, R. et al. | British Library Conference Proceedings | 2003