The invention provides an automatic driving vehicle control method and system based on multi-modal data fusion. The method comprises the following steps: acquiring a multi-modal image set including a three-view camera image, a long-distance camera image and a laser radar image in a vehicle driving environment; inputting the multi-modal image set into a data fusion network to obtain a vehicle driving control instruction; the data fusion network comprises a signal lamp sensing module, a scene sensing module and a trajectory prediction module; inputting the three-view camera image and the long-distance camera image into a signal lamp sensing module, and extracting signal lamp sensing features; inputting the three-view camera image and the laser radar image into a scene sensing module, and extracting scene sensing features; and fusing the signal lamp sensing features and the scene sensing features into multi-model fusion features, inputting the multi-model fusion features into a trajectory prediction module to predict a vehicle displacement vector, and obtaining a vehicle driving control instruction based on the displacement vector. By enhancing the perception of the state of the traffic signal lamp, the perception and understanding ability of the automatic driving system is improved.

    本发明提出一种基于多模态数据融合的自动驾驶车辆控制方法及系统,方法为:获取车辆行驶环境中包括三视角相机图像、远距相机图像和激光雷达图像的多模态图像集;将多模态图像集输入数据融合网络,得到车辆行驶控制指令;所述数据融合网络包括信号灯感知模块、场景感知模块和轨迹预测模块;将三视角相机图像和远距相机图像输入信号灯感知模块,提取信号灯感知特征;将三视角相机图像和激光雷达图像输入场景感知模块,提取场景感知特征;将信号灯感知特征与场景感知特征进行融合为多模型融合特征,输入轨迹预测模块预测车辆位移向量,基于位移向量得到车辆行驶控制指令。通过增强对交通信号灯状态的感知,提升自动驾驶系统的感知理解能力。


    Access

    Download


    Export, share and cite



    Title :

    Automatic driving vehicle control method and system based on multi-modal data fusion


    Additional title:

    基于多模态数据融合的自动驾驶车辆控制方法及系统


    Contributors:
    LI SHUANG (author)

    Publication date :

    2025-05-23


    Type of media :

    Patent


    Type of material :

    Electronic Resource


    Language :

    Chinese


    Classification :

    IPC:    B60W CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION , Gemeinsame Steuerung oder Regelung von Fahrzeug-Unteraggregaten verschiedenen Typs oder verschiedener Funktion



    Automatic driving method and system based on multi-view multi-modal fusion

    LIU CHUNSHENG / SUN JINYING / CHANG FALIANG et al. | European Patent Office | 2024

    Free access

    End-to-end automatic driving multi-modal data fusion method and system

    SU XIAOCONG / LONG KUN / CHEN JUEXUAN et al. | European Patent Office | 2025

    Free access


    Automatic driving safety assisting method and system based on multi-modal fusion

    CHEN LIN / LIANG HONGBIN / SHANG MINGSHENG et al. | European Patent Office | 2025

    Free access

    Vehicle automatic driving control method and system based on fusion

    CHENG DEXIN / ZHOU FENGMING / FU YUE et al. | European Patent Office | 2022

    Free access