The invention relates to the technical field of vehicles, and discloses a driver state evaluation method and system based on multi-mode signal monitoring, electronic equipment, a storage medium and a vehicle, and the method comprises the steps that multi-mode signals of a driver are collected in real time, and the multi-mode signals comprise video signals, audio signals, electrocardiosignals and vehicle state signals; the multi-modal signals are preprocessed, key features are extracted, multi-modal features are obtained, and the multi-modal features comprise driver and driving behavior features; the multi-modal features are fused to obtain fused features, and the fused features are used for describing the state of the driver; and inputting the fusion feature into a preset model to divide an abnormal state of the driver, and when the abnormal state exceeds a preset condition, correspondingly reminding the driver. The state of the driver can be detected in a multi-dimensional mode, the accuracy of state evaluation is improved, reminding can be correspondingly carried out, and driving safety is guaranteed.

    本发明涉及车辆技术领域,公开了一种基于多模态信号监测的驾驶员状态评估方法、系统、电子设备、存储介质及车辆,包括实时采集驾驶员的多模态信号,其中,所述多模态信号包括视频信号、音频信号、心电信号以及车辆状态信号;对所述多模态信号进行预处理,并进行关键特征提取,得到多模态特征,其中,所述多模态特征包括驾驶员以及驾驶行为特征;将所述多模态特征进行融合,得到融合特征,其中,所述融合特征用于描述驾驶员状态;将所述融合特征输入到预设的模型中,以划分驾驶员的异常状态,当所述异常状态超过预设条件时,对所述驾驶员进行相应提醒。本发明可多维度的检测驾驶员状态,提升状态评估的准确性,并可相应进行提醒,保障驾驶安全。


    Access

    Download


    Export, share and cite



    Title :

    Driver state evaluation method and system based on multi-mode signal monitoring


    Additional title:

    一种基于多模态信号监测的驾驶员状态评估方法及系统


    Contributors:
    WU QIUJIE (author)

    Publication date :

    2024-12-17


    Type of media :

    Patent


    Type of material :

    Electronic Resource


    Language :

    Chinese


    Classification :

    IPC:    B60W CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION , Gemeinsame Steuerung oder Regelung von Fahrzeug-Unteraggregaten verschiedenen Typs oder verschiedener Funktion / A61B DIAGNOSIS , Diagnostik / G06F ELECTRIC DIGITAL DATA PROCESSING , Elektrische digitale Datenverarbeitung / G06N COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS , Rechnersysteme, basierend auf spezifischen Rechenmodellen



    Intelligent cockpit multi-mode man-machine interaction driver state monitoring and adjusting system

    ZANG LIZHUO / SHANG JIAHUI / LI QINGSEN et al. | European Patent Office | 2022

    Free access

    Driver state warning system and method of monitoring driver state

    WOO MYEONG NAM | European Patent Office | 2023

    Free access

    Driver state monitoring method and system based on multi-sensor fusion

    ZHANG YINGWU / TAO XUEXIN / ZHANG WEI | European Patent Office | 2020

    Free access

    DRIVER STATE WARNING SYSTEM AND METHOD OF MONITORING DRIVER STATE

    WOO MYEONG NAM | European Patent Office | 2021

    Free access

    DRIVER STATE MONITORING SYSTEM

    CHUN HYE YEUN | European Patent Office | 2019

    Free access