The invention relates to the technical field of automatic driving, and discloses an automatic driving method and system based on empirical playback constraint strategy optimization, and the method comprises the steps: initializing strategy neural network parameters and value neural network parameters; designing an overhead function and a reward function, and interacting the intelligent agent with the environment to obtain a track; the empirical pool collects the trajectory and the probability distribution of the strategy neural network on the trajectory, and extracts a batch of trajectory with a specified size from the empirical pool for training to obtain training data; the current strategy is evaluated by using the value neural network, the strategy neural network is jointly updated through a strategy gradient algorithm and trust domain constraints, the value neural network is updated by means of a mean square error loss function, and if the neural network meets performance requirements, the neural network is applied to automatic driving to obtain the automatic driving method. According to the method, the RECPO algorithm is applied to expressway vehicle control, and a state space including the vehicle distance and the vehicle speed and an action space for vehicle speed adjustment and lane changing control are designed.
本发明涉及自动驾驶技术领域,公开了基于经验回放约束策略优化的自动驾驶方法和系统,方法包括初始化策略神经网络参数和价值神经网络参数;设计开销函数和奖励函数,智能体与环境交互,获得轨迹;经验池收集轨迹和策略神经网络对轨迹的概率分布,从经验池中抽取指定大小批次的轨迹进行训练,获得训练数据;利用价值神经网络来评估当前策略,并通过策略梯度算法和信任域约束共同更新策略神经网络,借助均方误差损失函数来更新价值神经网络,如果神经网络达到性能要求,将其应用于自动驾驶,得到自动驾驶方法。本发明将提出的RECPO算法应用于高速公路车辆控制,设计了包括车辆距离和车速在内的状态空间,以及车速调整和变道控制的动作空间。
Automatic driving method and system based on experience playback constraint strategy optimization
基于经验回放约束策略优化的自动驾驶方法和系统
2024-07-30
Patent
Electronic Resource
Chinese
IPC: | G06N COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS , Rechnersysteme, basierend auf spezifischen Rechenmodellen / B60W CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION , Gemeinsame Steuerung oder Regelung von Fahrzeug-Unteraggregaten verschiedenen Typs oder verschiedener Funktion |
Automatic driving multi-sensor synchronous playback method and system
European Patent Office | 2023
|TD3 unmanned driving decision-making method based on multi-experience playback pool
European Patent Office | 2024
|Vehicle automatic driving identification data playback method, system and device and storage medium
European Patent Office | 2022
|European Patent Office | 2023
|