Natural Human-Robot Interaction (HRI) is one of the key components for service robots to be able to work in human-centric environments. In such dynamic environments, the robot needs to understand the intention of the user to accomplish a task successfully. Towards addressing this point, we propose a software architecture that segments a target object from a crowded scene, indicated verbally by a human user. At the core of our system, we employ a multi-modal deep neural network for visual grounding. Unlike most grounding methods that tackle the challenge using pre-trained object detectors via a two-stepped process, we develop a single stage zero-shot model that is able to provide predictions in unseen data. We evaluate the performance of the proposed model on real RGB-D data collected from public scene datasets. Experimental results showed that the proposed model performs well in terms of accuracy and speed, while showcasing robustness to variation in the natural language input.


    Access

    Download


    Export, share and cite



    Title :

    Few-Shot Visual Grounding for Natural Human-Robot Interaction


    Contributors:

    Publication date :

    2021-05-18


    Remarks:

    Tziafas , G & Mohades Kasaei , S 2021 , Few-Shot Visual Grounding for Natural Human-Robot Interaction . in IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC) . IEEE , pp. 50-56 , 2021 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC) , Felra , Portugal , 28/04/2021 . https://doi.org/10.1109/ICARSC52212.2021.9429801



    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English


    Classification :

    DDC:    004 / 629



    Visual recognition of pointing gestures for human-robot interaction

    Nickel, K. / Stiefelhagen, R. | British Library Online Contents | 2007


    Visual grounding device for rail traffic

    WANG YUQIAO / XU HAN / YANG XUEFENG et al. | European Patent Office | 2015

    Free access

    GazeEMD : detecting visual intention in gaze-based human-robot interaction

    Shi, Lei / Copot, Cosmin / Vanlanduit, Steve | BASE | 2021

    Free access


    Human-Robot Interaction

    Sandor, Aniko / Cross, E. Vincent, II / Chang, Mai Lee | NTRS | 2015