The invention provides a multi-modal perception-based driver state detection method and system, and relates to the technical field of detection, and the method comprises the steps: obtaining the image information of a target cab, generating multi-modal driving image information, obtaining a driving fixation region sequence, and obtaining a driving control time-varying sequence; based on the driving fixation area sequence and the driving control time-varying sequence, carrying out driving behavior correlation analysis to obtain a control node correlation degree sequence; and finally, generating a first state check result according to the control node association degree sequence. Interacting with the target vehicle to obtain real-time physiological information of the target driver; and performing driving state complementation optimization of the first state detection result by adopting the real-time physiological information to obtain a target state detection result. The technical problem that in the prior art, due to single-mode perception analysis, the intervention effect of a detection result on dangerous driving of a driver is weak is solved, the intervention effect of the detection result on dangerous driving of the driver is enhanced, and the technical effects of accurate detection and traffic accident reduction are achieved.

    本发明提供了基于多模态感知的驾驶员状态检测方法及系统,涉及检测技术领域,通过获取目标驾驶室图像信息,生成多模态驾驶图像信息,再获取驾驶注视区域序列和获取驾驶控制时变序列,并基于驾驶注视区域序列和驾驶控制时变序列进行驾驶行为关联性分析,获得控制节点关联度序列;最后根据控制节点关联度序列生成第一状态检查结果;然后交互目标车辆获得目标驾驶员的实时生理信息;采用实时生理信息进行第一状态检查结果的驾驶状态补全优化,获得目标状态检测结果。解决了现有技术单模态感知分析使得检测结果对于驾驶员危险驾驶的干预作用较弱的技术问题,增强了检测结果对驾驶员危险驾驶的干预作用,达到了精准检测,降低交通事故的技术效果。


    Access

    Download


    Export, share and cite



    Title :

    Driver state detection method and system based on multi-modal perception


    Additional title:

    基于多模态感知的驾驶员状态检测方法及系统


    Contributors:
    YAN JUN (author) / WANG YONGFEI (author) / ZHANG JUHUA (author)

    Publication date :

    2024-01-05


    Type of media :

    Patent


    Type of material :

    Electronic Resource


    Language :

    Chinese


    Classification :

    IPC:    B60W CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION , Gemeinsame Steuerung oder Regelung von Fahrzeug-Unteraggregaten verschiedenen Typs oder verschiedener Funktion / G06V



    Computer Vision-Based Multi-Modal Advanced Driver Assistance System

    Chandolu, Suresh Babu / Medidhi, Naga Maneesha / Satyala, Chandanapriya et al. | IEEE | 2025


    State estimation method and system for multi-modal perception of foot robot

    WU HONGMIN / TANG GUANRONG / SU ZERONG et al. | European Patent Office | 2020

    Free access

    Intelligent traffic management method and system based on multi-modal perception

    YAN JUN / WANG YONGFEI / ZHANG JUHUA | European Patent Office | 2023

    Free access

    Driver emotion recognition method based on multi-modal data fusion

    WU CHAO / FENG CHENGJUN / ZHANG XIONG et al. | European Patent Office | 2025

    Free access

    Intelligent power domain control system based on driver state perception

    YANG SHICHUN / ZHANG BO'AO / CAO YAOGUANG et al. | European Patent Office | 2024

    Free access