An intriguing challenge in the human–robot interaction field is the prospect of endowing robots with emotional intelligence to make the interaction more genuine, intuitive, and natural. A crucial aspect in achieving this goal is the robot’s capability to infer and interpret human emotions. Thanks to its design and open programming platform, the NAO humanoid robot is one of the most widely used agents for human interaction. As with person-to-person communication, facial expressions are the privileged channel for recognizing the interlocutor’s emotional expressions. Although NAO is equipped with a facial expression recognition module, specific use cases may require additional features and affective computing capabilities that are not currently available. This study proposes a highly accurate convolutional-neural-network-based facial expression recognition model that is able to further enhance the NAO robot’ awareness of human facial expressions and provide the robot with an interlocutor’s arousal level detection capability. Indeed, the model tested during human–robot interactions was 91% and 90% accurate in recognizing happy and sad facial expressions, respectively; 75% accurate in recognizing surprised and scared expressions; and less accurate in recognizing neutral and angry expressions. Finally, the model was successfully integrated into the NAO SDK, thus allowing for high-performing facial expression classification with an inference time of 0.34 ± 0.04 s.


    Access

    Download


    Export, share and cite



    Title :

    Improving Human-Robot Interaction by Enhancing NAO Robot Awareness of Human Facial Expression



    Publication date :

    2021-01-01



    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English



    Classification :

    DDC:    629



    Improving human-robot interaction recognition

    Jayakanthana, K. R. | TIBKAT | 2023


    Effects of robot facial characteristics and gender in persuasive human-robot interaction

    Ghazali, Aimi S. / Ham, Jaap / Barakova, Emilia I. et al. | BASE | 2018

    Free access

    Evaluation of Human-Robot Interaction Awareness in Search and Rescue

    J. Scholtz / J. Young / H. A. Yanco et al. | NTIS | 2006


    Human-Robot Interaction Efficiency and Human-Robot Collaboration

    Galin, Rinat R. / Meshcheryakov, Roman V. | Springer Verlag | 2020


    Human-Robot Interaction

    Sandor, Aniko / Cross, E. Vincent, II / Chang, Mai Lee | NTRS | 2015