Real-time recognition of driver emotions can greatly improve traffic safety. With the rapid development of communication technology, it becomes possible to process large amounts of video data and identify the driver’s emotions in real time. To effectively recognize driver’s emotions, this paper proposes a new deep learning framework called Convolution Bidirectional Long Short-term Memory Neural Network (CBLNN). This method predicts the driver’s emotion based on the geometric features extracted from facial skin information and the heart rate extracted from changes in RGB components. The facial geometry features obtained by using Convolutional Neural Network (CNN) are intermediate variables for the heart rate analysis of Bidirectional Long Short Term Memory (Bi-LSTM). Subsequently, the output of Bi-LSTM is used as input to the CNN module to extract the hear rate features. CBLNN uses Multi-modal factorized bilinear pooling (MFB) to fuse the extracted information and classifies it into five common emotions: happiness, anger, sadness, fear and neutrality. Our emotion recognition method was tested, proving that it can be used to quickly and steadily recognize emotions in real time.
A Convolution Bidirectional Long Short-Term Memory Neural Network for Driver Emotion Recognition
IEEE Transactions on Intelligent Transportation Systems ; 22 , 7 ; 4570-4578
2021-07-01
2795734 byte
Aufsatz (Zeitschrift)
Elektronische Ressource
Englisch
Taylor & Francis Verlag | 2023
|