The concept of Industry 4.0 brings the change of industry manufacturing patterns that become more efficient and more flexible. In response to this tendency, an efficient robot teaching approach without complex programming has become a popular research direction. Therefore, we propose an interactive finger-touch based robot teaching schema using a multimodal 3D image (color (RGB), thermal (T) and point cloud (3D)) processing. Here, the resulting heat trace touching the object surface will be analyzed on multimodal data, in order to precisely identify the true hand/object contact points. These identified contact points are used to calculate the robot path directly. To optimize the identification of the contact points we propose a calculation scheme using a number of anchor points which are first predicted by hand/object point cloud segmentation. Subsequently a probability density function is defined to calculate the prior probability distribution of true finger trace. The temperature in the neighborhood of each anchor point is then dynamically analyzed to calculate the likelihood. Experiments show that the trajectories estimated by our multimodal method have significantly better accuracy and smoothness than only by analyzing point cloud and static temperature distribution.


    Zugriff

    Download


    Exportieren, teilen und zitieren



    Titel :

    Interactive robot teaching based on finger trajectory using multimodal RGB-D-T-data


    Beteiligte:
    Zhang, Yan (Autor:in) / Fütterer, Richard (Autor:in) / Notni, Gunther (Autor:in)

    Erscheinungsdatum :

    16.03.2023



    Medientyp :

    Aufsatz (Zeitschrift)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch



    Klassifikation :

    DDC:    004 / 629




    Interactive Multimodal Mobile Robot for Musical Performance

    Suzuki, K. / Ohashi, T. / Hashimoto, S. | British Library Conference Proceedings | 1999


    MULTIMODAL TRAJECTORY PREDICTIONS BASED ON GEOMETRIC ANCHORING

    ABDULHAMID ADAM / KURUTACH THANARD | Europäisches Patentamt | 2024

    Freier Zugriff

    TRAJECTORY-BASED MULTIMODAL TRANSPORT MANAGEMENT FOR RESILIENT TRANSPORTATION

    Evelin ENGLER / Stefan GEWIES / Paweł BANYŚ et al. | DOAJ | 2018

    Freier Zugriff

    Visual Interactive Trajectory Design

    Mathur, Ravishankar | TIBKAT | 2019