This work presents a framework for Human-Robot Collaboration (HRC) in assembly tasks that uses multimodal sensors, perception and control methods. First, vision sensing is employed for user identification to determine the collaborative task to be performed. Second, assembly actions and hand gestures are recognised using wearable inertial measurement units (IMUs) and convolutional neural networks (CNN) to identify when robot collaboration is needed and bring the next object to the user for assembly. If collaboration is not required, then the robot performs a solo task. Third, the robot arm uses time domain features from tactile sensors to detect when an object has been touched and grasped for handover actions in the assembly process. These multimodal sensors and computational modules are integrated in a layered control architecture for HRC collaborative assembly tasks. The proposed framework is validated in real-time using a Universal Robot arm (UR3) to collaborate with humans for assembling two types of objects 1) a box and 2) a small chair, and to work on a solo task of moving a stack of Lego blocks when collaboration with the user is not needed. The experiments show that the robot is capable of sensing and perceiving the state of the surrounding environment using multimodal sensors and computational methods to act and collaborate with humans to complete assembly tasks successfully.
Multimodal sensor-based human-robot collaboration in assembly tasks
2022-10-31
Male , J , Al , G A , Shabani , A & Martinez-Hernandez , U 2022 , Multimodal sensor-based human-robot collaboration in assembly tasks . in 2022 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2022 - Proceedings . Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics , vol. 2022-October , IEEE , U. S. A. , pp. 1266-1271 , 2022 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2022 , Prague , Czech Republic , 9/10/22 . https://doi.org/10.1109/SMC53654.2022.9945532
Aufsatz (Zeitschrift)
Elektronische Ressource
Englisch
assembly tasks , human-robot collaboration , vision and touch sensing , wearable sensing , /dk/atira/pure/subjectarea/asjc/2200/2208 , name=Electrical and Electronic Engineering , /dk/atira/pure/subjectarea/asjc/2200/2207 , name=Control and Systems Engineering , /dk/atira/pure/subjectarea/asjc/1700/1709 , name=Human-Computer Interaction
DDC: | 629 |
Balancing and scheduling assembly lines with human-robot collaboration tasks
BASE | 2022
|Balancing and scheduling assembly lines with human-robot collaboration tasks
BASE | 2022
|Safe Multimodal Communication in Human-Robot Collaboration
Springer Verlag | 2024
|