The invention discloses a learning and fusion method based on multi-modal track depth representation, and the method comprises the steps: constructing a multi-modal semantic track, and taking the multi-modal semantic track as the input of an encoder; constructing a dual-channel coding-single-channel decoding framework, and inputting the multi-modal semantic trajectory into the dual-channel coding-single-channel decoding framework; performing track embedding and text embedding on each track point of the multi-modal semantic track, outputting track word vector representation and text word vector representation, injecting position information, and outputting track word vector representation and text word vector representation containing position features; the track word vector representation and the text word vector representation of each track point are constrained, a track feature vector and a text feature vector are obtained, semantic information interaction is carried out through fine-grained alignment, and a fusion feature vector is obtained; according to the method, the problems of insufficient cross-modal data interaction and inconsistent feature vectors are solved, and the accuracy and authenticity of trajectory prediction are improved.

    本发明公开了一种基于多模态轨迹深度表示学习及融合方法,构建多模态语义轨迹,作为编码器的输入;构建双通道编码‑单通道解码框架,将多模态语义轨迹输入双通道编码‑单通道解码框架;对多模态语义轨迹的每个轨迹点进行轨迹嵌入和文本嵌入,输出轨迹词向量表征和文本词向量表征,并将位置信息注入,输出含有位置特征的轨迹词向量表征和文本词向量表征;对每个轨迹点的轨迹词向量表征和文本词向量表征进行约束,获取轨迹特征向量和文本特征向量,以细粒度对齐进行语义信息交互,获取融合特征向量;将融合特征向量输入解码器,输出预测轨迹,该方法突破了跨模态数据交互不充分及特征向量不一致问题,提高了轨迹预测的准确性和真实性。


    Zugriff

    Download


    Exportieren, teilen und zitieren



    Titel :

    Depth representation learning and fusion method based on multi-modal trajectory


    Weitere Titelangaben:

    一种基于多模态轨迹深度表示学习及融合方法


    Beteiligte:
    CHEN JING (Autor:in) / YUAN CHANGWEI (Autor:in) / ZHAO JIAO (Autor:in) / FENG JIAN (Autor:in) / WANG YU (Autor:in)

    Erscheinungsdatum :

    2023-10-27


    Medientyp :

    Patent


    Format :

    Elektronische Ressource


    Sprache :

    Chinesisch


    Klassifikation :

    IPC:    G06F ELECTRIC DIGITAL DATA PROCESSING , Elektrische digitale Datenverarbeitung / G06N COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS , Rechnersysteme, basierend auf spezifischen Rechenmodellen / G08G Anlagen zur Steuerung, Regelung oder Überwachung des Verkehrs , TRAFFIC CONTROL SYSTEMS



    Multi-modal trajectory prediction method

    JIANG WENJUAN / JIN ZHI / WANG REN et al. | Europäisches Patentamt | 2023

    Freier Zugriff

    Multi-modal multi-agent trajectory prediction

    SUN PEI / ZHAO HANG / MCCAULEY ALEXANDER et al. | Europäisches Patentamt | 2024

    Freier Zugriff

    MULTI-MODAL MULTI-AGENT TRAJECTORY PREDICTION

    SUN PEI / ZHAO HANG / MCCAULEY ALEXANDER et al. | Europäisches Patentamt | 2022

    Freier Zugriff


    Interactive vehicle multi-modal trajectory prediction method based on GRU-GCN

    MENG ZHIWEI / ZHANG SUMIN / HE RUI et al. | Europäisches Patentamt | 2023

    Freier Zugriff