To enable unmanned aerial vehicle (UAV) operators to efficiently and intuitively convey their commands to a swarm of UAVs, we propose the use of natural and human-centric input modalities, such as voices and gestures. This paper addresses the fusion of input modalities such as voice and gesture data, which are captured through a microphone and a Leap Motion controller, respectively, to control UAV swarms. The obtained experimental results are presented, and the achieved performance (accuracy) is analyzed. Finally, combined human factor ergonomics test with a questionnaire to verify the method’s validity.


    Zugriff

    Download


    Exportieren, teilen und zitieren



    Titel :

    Multimodal Fusion of Voice and Gesture Data for UAV Control


    Beteiligte:
    Xiaojia Xiang (Autor:in) / Qin Tan (Autor:in) / Han Zhou (Autor:in) / Dengqing Tang (Autor:in) / Jun Lai (Autor:in)


    Erscheinungsdatum :

    2022




    Medientyp :

    Aufsatz (Zeitschrift)


    Format :

    Elektronische Ressource


    Sprache :

    Unbekannt




    Multimodal Control of UAV Based on Gesture, Eye Movement and Voice Interaction

    Zhou, Ao / Han, Lei / Meng, Yuzhen | Springer Verlag | 2023


    Multimodal Fusion-GMM based Gesture Recognition for Smart Home by WiFi Sensing

    Ding, Jianyang / Wang, Yong / Si, Hongyan et al. | IEEE | 2022


    VEHICLE CONTROL APPARATUS AND METHOD USING VOICE OR GESTURE

    PARK SUN HONG / OH YOUNG DAL / KIM EUN JEONG | Europäisches Patentamt | 2015

    Freier Zugriff

    Gesture Recognition in Robotic Surgery with Multimodal Attention

    Van Amsterdam, B / Funke, I / Edwards, E et al. | BASE | 2022

    Freier Zugriff

    Multimodal Data Fusion for Big Events

    Papacharalampous, Alexandros E. / Cats, Oded / Lankhaar, Jan-Willem et al. | Transportation Research Record | 2016