The invention provides an automatic driving track decision method and device for a collaborative end-to-end large model, and the method comprises the steps: carrying out the processing of an RGB image of an own vehicle end, an RGB image of another vehicle end, and an RGB image of a road end through a first feature extraction network, and obtaining a first image feature, a second image feature, and a third image feature; performing fusion processing on the first image feature, the second image feature and the third image feature to obtain an image fusion feature; processing the point cloud data of the road end by using a second feature extraction network to obtain a road end point cloud feature; processing the image fusion feature and the road end point cloud feature to obtain a first BEV feature and a second BEV feature; fusing the first BEV feature and the second BEV feature to obtain a fused BEV feature; fusing the prompt information and the fused BEV features by using a large visual language model to obtain text information; and processing the text information by using the natural language large model to obtain a self-vehicle trajectory decision result. According to the invention, the accuracy of trajectory decision is improved.

    本申请提供面向协同端到端大模型的自动驾驶轨迹决策方法及装置,其中的方法包括:利用第一特征提取网络对自车端的RGB图像、他车端的RGB图像和路端的RGB图像进行处理,得到第一图像特征、第二图像特征和第三图像特征;对第一图像特征、第二图像特征和第三图像特征进行融合处理得到图像融合特征;利用第二特征提取网络对路端的点云数据进行处理得到路端点云特征;对图像融合特征和路端点云特征进行处理得到第一BEV特征和第二BEV特征;对第一BEV特征和第二BEV特征进行融合得到融合BEV特征;用大视觉语言模型将prompt信息和融合BEV特征进行融合得到文本信息;利用自然语言大模型对文本信息进行处理得到自车轨迹决策结果。本申请提高了轨迹决策的准确性。


    Zugriff

    Download


    Exportieren, teilen und zitieren



    Titel :

    Collaborative end-to-end large model-oriented automatic driving track decision method and device


    Weitere Titelangaben:

    面向协同端到端大模型的自动驾驶轨迹决策方法及装置


    Beteiligte:
    LU FENGLI (Autor:in) / LI ZHIWEI (Autor:in) / WANG JINGWEI (Autor:in) / GUO YIPIAO (Autor:in) / WANG KUNFENG (Autor:in) / SHEN TIANYU (Autor:in) / WANG LI (Autor:in) / TAN QIFAN (Autor:in) / ZHANG XINYU (Autor:in)

    Erscheinungsdatum :

    2024-05-14


    Medientyp :

    Patent


    Format :

    Elektronische Ressource


    Sprache :

    Chinesisch


    Klassifikation :

    IPC:    B60W CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION , Gemeinsame Steuerung oder Regelung von Fahrzeug-Unteraggregaten verschiedenen Typs oder verschiedener Funktion / G06F ELECTRIC DIGITAL DATA PROCESSING , Elektrische digitale Datenverarbeitung / G06V



    Seat adjustment and automatic driving decision collaborative optimization system and method

    SHEN CHUANLIANG / ZHANG LONGXU / MA XIAOYUAN et al. | Europäisches Patentamt | 2024

    Freier Zugriff

    Intersection multi-object participation-oriented automatic driving decision-making method

    CHEN KEYU / ZHANG HAN / YE YULIN | Europäisches Patentamt | 2022

    Freier Zugriff

    Driving track prediction method and device and automatic driving vehicle

    HAN ZHIHUA / XU XIUXIN / HUANG WENHUAN | Europäisches Patentamt | 2021

    Freier Zugriff


    Collaborative automatic driving system

    RAN BIN / CHEN TIANYI / LI SHEN et al. | Europäisches Patentamt | 2023

    Freier Zugriff