The invention discloses an automatic driving key scene generation method based on multi-agent reinforcement learning, and the method employs a heterogeneous multi-agent depth deterministic strategy gradient, and can process that different agents have different action spaces and observation spaces, and each agent has an own strategy network model and a target network (a multi-task value network). The ability of multi-agent interaction processing and dynamic environment adaptability in a simulation driving scene is improved, and a reasonable and interactive traffic scene can be generated. For different reward function task evaluation of heterogeneous multi-agents, a novel multi-task value network architecture is established, and behavior values of various agents (such as traffic vehicles and target vehicles) under different task situations are effectively evaluated by sharing a hidden layer feature extraction layer and matching with a task-specific value function header. Meanwhile, a strategy network model is established, a recurrent neural network meaning mechanism is introduced, and the network structure can be generalized to a scene which is not encountered by learning a nonlinear relation between input and output, so that the intelligent agent has the capability of flexibly adjusting a driving strategy under different conditions.
本发明公开了一种基于多智能体强化学习的自动驾驶关键场景生成方法,采用异构多智能体深度确定性策略梯度,能够处理不同智能体具有不同行动空间和观测空间,每个智能体都有自己的策略网络模型和目标网络(多任务价值网络)。提高了仿真驾驶场景中多智能体交互处理、动态环境适应性的能力,可以生成合理可交互的交通场景。针对异构多智能体的差异化奖励函数任务评估,本发明建立了一种新颖的多任务价值网络架构,通过共享隐藏层特征提取层并配以任务特定的价值函数头部,有效地评估了在不同任务情境下各类智能体(如交通车与目标车)的行为价值。同时本发明建立了一种策略网络模型,引入了循环神经网络意力机制,通过学习输入与输出之间的非线性关系,该网络结构能够泛化到未曾遇到过的场景,使智能体具备在不同情况下灵活调整行驶策略的能力。
Automatic driving key scene generation method based on multi-agent reinforcement learning
一种基于多智能体强化学习的自动驾驶关键场景生成方法
2024-08-09
Patent
Electronic Resource
Chinese
IPC: | G06F ELECTRIC DIGITAL DATA PROCESSING , Elektrische digitale Datenverarbeitung / B60W CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION , Gemeinsame Steuerung oder Regelung von Fahrzeug-Unteraggregaten verschiedenen Typs oder verschiedener Funktion / G06N COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS , Rechnersysteme, basierend auf spezifischen Rechenmodellen |
Automatic driving vehicle control method based on multi-agent reinforcement learning
European Patent Office | 2023
|Automatic driving solution under multi-target complex traffic scene based on reinforcement learning
European Patent Office | 2022
|European Patent Office | 2023
|European Patent Office | 2024
|European Patent Office | 2024
|