The invention discloses a driver state monitoring method based on multi-sensor fusion. The method comprises the steps: carrying out zooming of an obtained image at different scales, generating an image pyramid, and carrying out the positioning of a human face and five sense organs; as for the human face and the five sense organs obtained through positioning, extracting eye and mouth area featuresthrough a convolutional neural network, performing statistics on the eye closing duration within a certain period of time, if the eye closing duration exceeds a threshold value, judging that the driver is in a first fatigue state, performing statistics on the mouth opening duration within a certain period of time, if the mouth opening duration exceeds the threshold value, judging that the driver is in a second fatigue state; collecting heart rate information of the driver, calculating a heart rate variance D through a continuous time queue, and when D is less than a threshold value T, judgingjudged that the driver is in a third fatigue state; and setting the weights of the three fatigue states, superposing the three fatigue states, judging the fatigue level, and reminding in a voice broadcast and/or seat vibration mode. The face recognition result and the heart rate monitoring result are dynamically fused, the fatigue driving state can be accurately judged and the precision is higher.

    本发明公开了一种基于多传感器融合的驾驶员状态监测方法,包括:将获取的图像进行不同尺度的缩放,生成图像金字塔,对人脸和五官进行定位;对定位得到的人脸和五官,通过卷积神经网络提取眼部和嘴部区域特征,统计一定时间内闭眼的时长,若超过阈值,则判别处于第一疲劳状态,统计一定时间嘴巴张开的时长,若超过阈值,则判别处于第二疲劳状态;采集驾驶员的心率信息,通过连续时间的队列计算心率方差D,当D<阈值T时,则判断处于第三疲劳状态;设置三种疲劳状态的权重,将三种疲劳状态进行叠加,判定疲劳等级,并以语音播报和/或震动座椅的方式进行提醒。将面部识别结果和心率监测结果进行动态融合,可以准确判定疲劳驾驶状态,精度更高。


    Access

    Download


    Export, share and cite



    Title :

    Driver state monitoring method and system based on multi-sensor fusion


    Additional title:

    基于多传感器融合的驾驶员状态监测方法和系统


    Contributors:
    ZHANG YINGWU (author) / TAO XUEXIN (author) / ZHANG WEI (author)

    Publication date :

    2020-05-19


    Type of media :

    Patent


    Type of material :

    Electronic Resource


    Language :

    Chinese


    Classification :

    IPC:    G08B SIGNALLING OR CALLING SYSTEMS , Signalisier- oder Rufsysteme / B60W CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION , Gemeinsame Steuerung oder Regelung von Fahrzeug-Unteraggregaten verschiedenen Typs oder verschiedener Funktion / G06K Erkennen von Daten , RECOGNITION OF DATA



    MULTI-SENSOR FUSION-BASED DRIVER MONITORING APPARATUS AND METHOD

    HAN DONGSEOG / ALWIN POULOSE | European Patent Office | 2023

    Free access

    MULTI SENSOR FUSION-BASED DRIVER MONITORING DEVICE AND METHOD

    DONG SEOG HAN / ALWIN POULOSE | European Patent Office | 2023

    Free access

    MULTI SENSOR FUSION-BASED DRIVER MONITORING DEVICE AND METHOD

    European Patent Office | 2023

    Free access


    Multi-sensor driver drowsiness monitoring

    Boyraz,P. / Acar,M. / Kerr,D. et al. | Automotive engineering | 2008