The embodiment of the invention provides a multi-modal data fusion method and device for driver behavior prediction. The multi-modal data fusion method for driver behavior prediction comprises the steps of obtaining initial data; wherein the initial data comprises image data, audio data and vehicle state data; determining a data preprocessing rule based on the category of the initial data, processing the initial data based on the data preprocessing rule, and determining a multi-modal feature; and performing data fusion based on the category of the multi-modal features, and determining fusion features. The method comprises: acquiring initial data; determining a data preprocessing rule based on the category of the initial data, processing the initial data based on the data preprocessing rule, and determining a multi-modal feature; according to the method, data fusion is performed based on the category of the multi-modal features, and the fusion features are determined, so that information of different data sources can be integrated together, and the features of driver behaviors can be captured more comprehensively and accurately.
本说明书实施例提供用于司机行为预测的多模态数据融合方法及装置,其中用于司机行为预测的多模态数据融合方法包括:获取初始数据;其中,初始数据包括图像数据、音频数据和车辆状态数据;基于初始数据的类别确定数据预处理规则,并基于数据预处理规则对初始数据进行处理,确定多模态特征;基于多模态特征的类别进行数据融合,确定融合特征。通过获取初始数据;基于初始数据的类别确定数据预处理规则,并基于数据预处理规则对初始数据进行处理,确定多模态特征;基于多模态特征的类别进行数据融合,确定融合特征,可以实现将不同数据源的信息整合在一起,以更全面和准确地捕捉驾驶员行为的特征。
Multi-modal data fusion method and device for driver behavior prediction
用于司机行为预测的多模态数据融合方法及装置
2024-11-08
Patent
Electronic Resource
Chinese
IPC: | G06F ELECTRIC DIGITAL DATA PROCESSING , Elektrische digitale Datenverarbeitung / B60W CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION , Gemeinsame Steuerung oder Regelung von Fahrzeug-Unteraggregaten verschiedenen Typs oder verschiedener Funktion / G06N COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS , Rechnersysteme, basierend auf spezifischen Rechenmodellen / G06V / G10L Analyse oder Synthese von Sprache , SPEECH ANALYSIS OR SYNTHESIS |
Driver emotion recognition method based on multi-modal data fusion
European Patent Office | 2025
|Multi-modal traffic flow prediction method based on multi-source data feature fusion
European Patent Office | 2025
|M2DA: Multi-Modal Fusion Transformer Incorporating Driver Attention for Autonomous Driving
ArXiv | 2024
|Semantic Analysis of Driver Behavior by Data Fusion
TIBKAT | 2020
|