The invention relates to an automatic driving safety assisting method and system based on multi-modal fusion, and belongs to the field of automatic driving. The system is composed of a multi-mode sensor, a preprocessing unit, a feature encoder, a time sequence merging module, a context attention module, a classification module, a data storage unit and an output unit. The method comprises the following steps: S1, collecting visual image data and non-visual image data; s2, preprocessing the visual image data and the non-visual image data; s3, performing feature extraction by using a feature encoder; s4, key event division is carried out by using a time sequence merging module; S5, feature fusion is carried out by using a context attention module; s6, inputting the fusion features into a classification module, and predicting pedestrian behavior types; and S7, storing a pedestrian behavior type prediction result, and feeding back the pedestrian behavior type prediction result to the automobile terminal system. According to the method, key events related to pedestrian behaviors can be effectively identified and aggregated, and the accuracy and efficiency of pedestrian behavior intention prediction are improved.

    本发明涉及基于多模态融合的自动驾驶安全性辅助方法和系统,属于自动驾驶领域。该系统由多模态传感器、预处理单元、特征编码器、时序合并模块、上下文注意力模块、分类模块、数据存储单元、输出单元构成。该方法包含以下步骤:S1:采集视觉图像数据和非视觉图像数据;S2:对视觉图像数据和非视觉图像数据进行预处理;S3:利用特征编码器进行特征提取;S4:利用时序合并模块进行关键事件划分:S5:利用上下文注意力模块对进行特征融合;S6:将融合特征输入分类模块中,预测行人行为类型;S7:将预测行人行为类型结果存储,并反馈给汽车终端系统。本发明方法能够有效地识别和聚合与行人行为相关的关键事件,提高行人行为意图预测的准确性和效率。


    Access

    Download


    Export, share and cite



    Title :

    Automatic driving safety assisting method and system based on multi-modal fusion


    Additional title:

    基于多模态融合的自动驾驶安全性辅助方法和系统


    Contributors:
    CHEN LIN (author) / LIANG HONGBIN (author) / SHANG MINGSHENG (author) / GONG ZHIGANG (author)

    Publication date :

    2025-02-21


    Type of media :

    Patent


    Type of material :

    Electronic Resource


    Language :

    Chinese


    Classification :

    IPC:    G06V / B60W CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION , Gemeinsame Steuerung oder Regelung von Fahrzeug-Unteraggregaten verschiedenen Typs oder verschiedener Funktion / G06F ELECTRIC DIGITAL DATA PROCESSING , Elektrische digitale Datenverarbeitung / G06N COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS , Rechnersysteme, basierend auf spezifischen Rechenmodellen



    AUTOMATIC DRIVING ASSISTING SYSTEM, AUTOMATIC DRIVING ASSISTING METHOD, AND COMPUTER PROGRAM

    KONDO YOSHITO / SATO YUJI / ISHIKAWA TAKESHI | European Patent Office | 2015

    Free access

    AUTOMATIC DRIVING ASSISTING SYSTEM, AUTOMATIC DRIVING ASSISTING METHOD, AND COMPUTER PROGRAM

    GOTO HIROHIKO / SATO YUJI / TANAKA KUNIAKI et al. | European Patent Office | 2016

    Free access

    Automatic driving method and system based on multi-view multi-modal fusion

    LIU CHUNSHENG / SUN JINYING / CHANG FALIANG et al. | European Patent Office | 2024

    Free access

    Driving safety assisting method and system

    HUANG XUNWEN | European Patent Office | 2023

    Free access

    End-to-end automatic driving multi-modal data fusion method and system

    SU XIAOCONG / LONG KUN / CHEN JUEXUAN et al. | European Patent Office | 2025

    Free access