Despite advances in robotic perception are increasing autonomous capabilities, human intelligence is still considered a necessity in unstructured or unpredictable environments. Hence, also according to the Industry 4.0 paradigm, humans and robots are encouraged to achieve mutual Human-Robot Interaction (HRI). HRI can be physical (pHRI) or not, depending on the assigned task. For example, when the robot is constrained in a dangerous environment or must handle hazardous materials, pHRI is not recommended. In these cases, robot teleoperation may be necessary. A teleoperation system concerns with the exploration and exploitation of spaces where the user presence is not allowed. Therefore, the operator needs to move the robot remotely. Although plenty of human-machine interfaces for teleoperation have been developed considering a mechanical device, vision-based interfaces do not require physical contact with external devices. This grants a more natural and intuitive interaction, which is reflected in task performance. Our proposed system is a novel robot teleoperation system that exploits RGB cameras, which are easy to use and commonly available on the market at a reduced price. A ROS-based framework has been developed to supply hand tracking and hand-gesture recognition features, exploiting the OpenPose software based on the Deep Learning framework Caffe. This, in combination with the ease of availability of an RGB camera, leads the framework to be strongly open-source-oriented and highly replicable on all ROS-based platforms. It is worth noting that the system does not include the Z-axis control in this first version. This is due to the high precision and sensitivity required to robustly control the third axis, a precision that 3D vision systems are not able to provide unless very expensive devices are adopted. Our aim is to further develop the system to include the third axis control in a future release.


    Access

    Download


    Export, share and cite



    Title :

    A vision-based teleoperation system for robotic systems



    Publication date :

    2020-01-01


    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English



    Classification :

    DDC:    629




    Systems and methods of remote teleoperation of robotic vehicles

    SUMMER MATTHEW D / BOWMAN WILLIAM S / FALENDYSZ ANDREW D et al. | European Patent Office | 2024

    Free access

    SYSTEMS AND METHODS OF REMOTE TELEOPERATION OF ROBOTIC VEHICLES

    SUMMER MATTHEW D / BOWMAN WILLIAM S / FALENDYSZ ANDREW D et al. | European Patent Office | 2022

    Free access

    Development and Teleoperation of Robotic Vehicles

    Kitts, Christopher / Quinn, N. / Ota, J. et al. | AIAA | 2003


    SYSTEMS AND METHODS OF REMOTE TELEOPERATION OF ROBOTIC VEHICLES

    SUMMER MATTHEW / BOWMAN WILLIAM / FALENDYSZ ANDREW et al. | European Patent Office | 2020

    Free access