This work presents a framework for Human-Robot Collaboration (HRC) in assembly tasks that uses multimodal sensors, perception and control methods. First, vision sensing is employed for user identification to determine the collaborative task to be performed. Second, assembly actions and hand gestures are recognised using wearable inertial measurement units (IMUs) and convolutional neural networks (CNN) to identify when robot collaboration is needed and bring the next object to the user for assembly. If collaboration is not required, then the robot performs a solo task. Third, the robot arm uses time domain features from tactile sensors to detect when an object has been touched and grasped for handover actions in the assembly process. These multimodal sensors and computational modules are integrated in a layered control architecture for HRC collaborative assembly tasks. The proposed framework is validated in real-time using a Universal Robot arm (UR3) to collaborate with humans for assembling two types of objects 1) a box and 2) a small chair, and to work on a solo task of moving a stack of Lego blocks when collaboration with the user is not needed. The experiments show that the robot is capable of sensing and perceiving the state of the surrounding environment using multimodal sensors and computational methods to act and collaborate with humans to complete assembly tasks successfully.


    Zugriff

    Download


    Exportieren, teilen und zitieren



    Titel :

    Multimodal sensor-based human-robot collaboration in assembly tasks


    Beteiligte:

    Erscheinungsdatum :

    2022-10-31


    Anmerkungen:

    Male , J , Al , G A , Shabani , A & Martinez-Hernandez , U 2022 , Multimodal sensor-based human-robot collaboration in assembly tasks . in 2022 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2022 - Proceedings . Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics , vol. 2022-October , IEEE , U. S. A. , pp. 1266-1271 , 2022 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2022 , Prague , Czech Republic , 9/10/22 . https://doi.org/10.1109/SMC53654.2022.9945532



    Medientyp :

    Aufsatz (Zeitschrift)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch



    Klassifikation :

    DDC:    629



    Multimodal Human-Robot Collaboration in Assembly

    Liu, Sichao | BASE | 2022

    Freier Zugriff

    Balancing and scheduling assembly lines with human-robot collaboration tasks

    Nourmohammadi, Amir / Fathi, Masood / Ng, Amos H. C. | BASE | 2022

    Freier Zugriff

    Balancing and scheduling assembly lines with human-robot collaboration tasks

    Nourmohammadi, Amir / Fathi, Masood / Ng, Amos H. C. | BASE | 2022

    Freier Zugriff

    Safe Multimodal Communication in Human-Robot Collaboration

    Ferrari, Davide / Pupa, Andrea / Signoretti, Alberto et al. | Springer Verlag | 2024


    Toward Multimodal Human-Robot Cooperation and Collaboration

    Perzanowski, Dennis / Brock, Derek / Bugajska, Magdalena et al. | AIAA | 2004