The embodiment of the invention provides a multi-modal data fusion method and device for driver behavior prediction. The multi-modal data fusion method for driver behavior prediction comprises the steps of obtaining initial data; wherein the initial data comprises image data, audio data and vehicle state data; determining a data preprocessing rule based on the category of the initial data, processing the initial data based on the data preprocessing rule, and determining a multi-modal feature; and performing data fusion based on the category of the multi-modal features, and determining fusion features. The method comprises: acquiring initial data; determining a data preprocessing rule based on the category of the initial data, processing the initial data based on the data preprocessing rule, and determining a multi-modal feature; according to the method, data fusion is performed based on the category of the multi-modal features, and the fusion features are determined, so that information of different data sources can be integrated together, and the features of driver behaviors can be captured more comprehensively and accurately.

    本说明书实施例提供用于司机行为预测的多模态数据融合方法及装置,其中用于司机行为预测的多模态数据融合方法包括:获取初始数据;其中,初始数据包括图像数据、音频数据和车辆状态数据;基于初始数据的类别确定数据预处理规则,并基于数据预处理规则对初始数据进行处理,确定多模态特征;基于多模态特征的类别进行数据融合,确定融合特征。通过获取初始数据;基于初始数据的类别确定数据预处理规则,并基于数据预处理规则对初始数据进行处理,确定多模态特征;基于多模态特征的类别进行数据融合,确定融合特征,可以实现将不同数据源的信息整合在一起,以更全面和准确地捕捉驾驶员行为的特征。


    Access

    Download


    Export, share and cite



    Title :

    Multi-modal data fusion method and device for driver behavior prediction


    Additional title:

    用于司机行为预测的多模态数据融合方法及装置


    Contributors:
    LI LIJIANG (author) / ZHAO XUE (author) / HU MIN (author) / LU BAOLI (author) / JIANG LIMIN (author) / NING XIN (author) / TANG XIAOJIANG (author) / LI SHUANG (author)

    Publication date :

    2024-11-08


    Type of media :

    Patent


    Type of material :

    Electronic Resource


    Language :

    Chinese


    Classification :

    IPC:    G06F ELECTRIC DIGITAL DATA PROCESSING , Elektrische digitale Datenverarbeitung / B60W CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION , Gemeinsame Steuerung oder Regelung von Fahrzeug-Unteraggregaten verschiedenen Typs oder verschiedener Funktion / G06N COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS , Rechnersysteme, basierend auf spezifischen Rechenmodellen / G06V / G10L Analyse oder Synthese von Sprache , SPEECH ANALYSIS OR SYNTHESIS



    Driver emotion recognition method based on multi-modal data fusion

    WU CHAO / FENG CHENGJUN / ZHANG XIONG et al. | European Patent Office | 2025

    Free access

    Multi-modal traffic flow prediction method based on multi-source data feature fusion

    CHEN LULU / LI SHIJIE / JIANG HUAIGUANG et al. | European Patent Office | 2025

    Free access

    M2DA: Multi-Modal Fusion Transformer Incorporating Driver Attention for Autonomous Driving

    Xu, Dongyang / Li, Haokun / Wang, Qingfan et al. | ArXiv | 2024

    Free access

    DRIVER BEHAVIOR PREDICTION DEVICE

    YAMAMOTO CHIHARU | European Patent Office | 2019

    Free access