The invention discloses a learning and fusion method based on multi-modal track depth representation, and the method comprises the steps: constructing a multi-modal semantic track, and taking the multi-modal semantic track as the input of an encoder; constructing a dual-channel coding-single-channel decoding framework, and inputting the multi-modal semantic trajectory into the dual-channel coding-single-channel decoding framework; performing track embedding and text embedding on each track point of the multi-modal semantic track, outputting track word vector representation and text word vector representation, injecting position information, and outputting track word vector representation and text word vector representation containing position features; the track word vector representation and the text word vector representation of each track point are constrained, a track feature vector and a text feature vector are obtained, semantic information interaction is carried out through fine-grained alignment, and a fusion feature vector is obtained; according to the method, the problems of insufficient cross-modal data interaction and inconsistent feature vectors are solved, and the accuracy and authenticity of trajectory prediction are improved.
本发明公开了一种基于多模态轨迹深度表示学习及融合方法,构建多模态语义轨迹,作为编码器的输入;构建双通道编码‑单通道解码框架,将多模态语义轨迹输入双通道编码‑单通道解码框架;对多模态语义轨迹的每个轨迹点进行轨迹嵌入和文本嵌入,输出轨迹词向量表征和文本词向量表征,并将位置信息注入,输出含有位置特征的轨迹词向量表征和文本词向量表征;对每个轨迹点的轨迹词向量表征和文本词向量表征进行约束,获取轨迹特征向量和文本特征向量,以细粒度对齐进行语义信息交互,获取融合特征向量;将融合特征向量输入解码器,输出预测轨迹,该方法突破了跨模态数据交互不充分及特征向量不一致问题,提高了轨迹预测的准确性和真实性。
Depth representation learning and fusion method based on multi-modal trajectory
一种基于多模态轨迹深度表示学习及融合方法
2023-10-27
Patent
Elektronische Ressource
Chinesisch
IPC: | G06F ELECTRIC DIGITAL DATA PROCESSING , Elektrische digitale Datenverarbeitung / G06N COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS , Rechnersysteme, basierend auf spezifischen Rechenmodellen / G08G Anlagen zur Steuerung, Regelung oder Überwachung des Verkehrs , TRAFFIC CONTROL SYSTEMS |
Interactive vehicle multi-modal trajectory prediction method based on GRU-GCN
Europäisches Patentamt | 2023
|