The invention relates to the technical field of automatic driving, and discloses an end-to-end automatic driving multi-modal data fusion method and system, and the method comprises the steps: obtaining multiple pieces of sensor data, GPS data and vehicle body data of an automatic driving vehicle, and the multiple pieces of sensor data comprise laser radar data, camera data and millimeter wave radar data; performing pre-fusion processing on the laser radar data and the camera data to obtain a pre-fusion result; and performing post-fusion processing on the pre-fusion result, the millimeter wave radar data, the GPS data and the vehicle body data to obtain a post-fusion result so as to realize multi-modal data fusion, and further realizing automatic driving decision through a large model technology based on the pre-fusion result and the post-fusion result. According to the invention, efficient fusion of multi-modal data in a complex driving environment is realized, the accuracy and reliability of automatic driving decision making are improved, and the stability and safety of an automatic driving system in a dynamic environment are also improved.

    本发明涉及自动驾驶技术领域,公开了一种端到端自动驾驶多模态数据融合方法及系统,所述方法包括:获取自动驾驶车辆的多个传感器数据、GPS数据以及车身数据,多个传感器数据包括激光雷达数据、相机数据以及毫米波雷达数据;对激光雷达数据和相机数据进行前融合处理,获得前融合结果;对前融合结果、毫米波雷达数据、GPS数据以及车身数据进行后融合处理,获得后融合结果,以实现多模态数据的融合,进一步地,基于所述前融合结果和所述后融合结果通过大模型技术实现自动驾驶决策,由此,实现了复杂驾驶环境下多模态数据的高效融合,提高了自动驾驶决策的准确性和可靠性,也提升了自动驾驶系统在动态环境下的稳定性与安全性。


    Zugriff

    Download


    Exportieren, teilen und zitieren



    Titel :

    End-to-end automatic driving multi-modal data fusion method and system


    Weitere Titelangaben:

    端到端自动驾驶多模态数据融合方法及系统


    Beteiligte:
    SU XIAOCONG (Autor:in) / LONG KUN (Autor:in) / CHEN JUEXUAN (Autor:in) / LU LUNLUN (Autor:in)

    Erscheinungsdatum :

    25.02.2025


    Medientyp :

    Patent


    Format :

    Elektronische Ressource


    Sprache :

    Chinesisch


    Klassifikation :

    IPC:    B60W CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION , Gemeinsame Steuerung oder Regelung von Fahrzeug-Unteraggregaten verschiedenen Typs oder verschiedener Funktion / G06F ELECTRIC DIGITAL DATA PROCESSING , Elektrische digitale Datenverarbeitung / G06N COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS , Rechnersysteme, basierend auf spezifischen Rechenmodellen



    Automatic driving method and system based on multi-view multi-modal fusion

    LIU CHUNSHENG / SUN JINYING / CHANG FALIANG et al. | Europäisches Patentamt | 2024

    Freier Zugriff

    Automatic driving safety assisting method and system based on multi-modal fusion

    CHEN LIN / LIANG HONGBIN / SHANG MINGSHENG et al. | Europäisches Patentamt | 2025

    Freier Zugriff

    Automatic driving model based on multi-modal large model, training method and automatic driving method

    HUANG JIZHOU / ZENG ZENGFENG | Europäisches Patentamt | 2025

    Freier Zugriff


    Multi-Modal Fusion for Enhanced Automatic Modulation Classification

    Li, Yingkai / Wang, Shufei / Zhang, Yibin et al. | IEEE | 2024