Existing visual-based fatigue detection methods usually monitor drivers’ fatigue by capturing their facial features, including eyelid movements, yawn frequency and head pose. However, these approaches typically do not take drivers’ biological signals into consideration. An accurate model for fatigue detection requires combining both facial behavior and biological data. This paper proposes a novel non-intrusive method for driver multimodal fusion fatigue detection by extracting eyelid features and heart rate signals from the RGB video. The multimodal feature fusion method could significantly increase the accuracy of fatigue detection. Specifically, we established two fatigue detection models based on heart rate and the PERCLOS value respectively with one-dimensional Convolutional Neural Network (1D CNN), where the PERCLOS refers to the percentage of eyelid closure over the pupil. Finally, the outputs of the two models are weighted to achieve the multimodal fusion fatigue detection. Simulation results show that our method yield better performance than traditional methods.
A Multimodal Fusion Fatigue Driving Detection Method Based on Heart Rate and PERCLOS
IEEE Transactions on Intelligent Transportation Systems ; 23 , 11 ; 21810-21820
2022-11-01
2156363 byte
Aufsatz (Zeitschrift)
Elektronische Ressource
Englisch
PERCLOS+: Moving Beyond Single-Metric Drowsiness Monitors
British Library Conference Proceedings | 2008
|Noninvasive Heart Rate Variability Detection Device for Fatigue Driving Detection System
Trans Tech Publications | 2012
|Fatigue Driving Detection Based on Multi Feature Fusion
IEEE | 2019
|Fatigue driving detection method based on multi-information fusion
Europäisches Patentamt | 2020
|