The invention provides an automatic driving track decision method and device for a collaborative end-to-end large model, and the method comprises the steps: carrying out the processing of an RGB image of an own vehicle end, an RGB image of another vehicle end, and an RGB image of a road end through a first feature extraction network, and obtaining a first image feature, a second image feature, and a third image feature; performing fusion processing on the first image feature, the second image feature and the third image feature to obtain an image fusion feature; processing the point cloud data of the road end by using a second feature extraction network to obtain a road end point cloud feature; processing the image fusion feature and the road end point cloud feature to obtain a first BEV feature and a second BEV feature; fusing the first BEV feature and the second BEV feature to obtain a fused BEV feature; fusing the prompt information and the fused BEV features by using a large visual language model to obtain text information; and processing the text information by using the natural language large model to obtain a self-vehicle trajectory decision result. According to the invention, the accuracy of trajectory decision is improved.
本申请提供面向协同端到端大模型的自动驾驶轨迹决策方法及装置,其中的方法包括:利用第一特征提取网络对自车端的RGB图像、他车端的RGB图像和路端的RGB图像进行处理,得到第一图像特征、第二图像特征和第三图像特征;对第一图像特征、第二图像特征和第三图像特征进行融合处理得到图像融合特征;利用第二特征提取网络对路端的点云数据进行处理得到路端点云特征;对图像融合特征和路端点云特征进行处理得到第一BEV特征和第二BEV特征;对第一BEV特征和第二BEV特征进行融合得到融合BEV特征;用大视觉语言模型将prompt信息和融合BEV特征进行融合得到文本信息;利用自然语言大模型对文本信息进行处理得到自车轨迹决策结果。本申请提高了轨迹决策的准确性。
Collaborative end-to-end large model-oriented automatic driving track decision method and device
面向协同端到端大模型的自动驾驶轨迹决策方法及装置
2024-05-14
Patent
Elektronische Ressource
Chinesisch
Seat adjustment and automatic driving decision collaborative optimization system and method
Europäisches Patentamt | 2024
|Intersection multi-object participation-oriented automatic driving decision-making method
Europäisches Patentamt | 2022
|Driving track prediction method and device and automatic driving vehicle
Europäisches Patentamt | 2021
|Automatic driving collaborative perception system and automatic driving control method and device
Europäisches Patentamt | 2024
|