A vehicle ramp entrance confluence control method based on deep reinforcement learning is characterized in that a SUMO is utilized to construct a ramp confluence simulation environment and obtain related state information, a state space, an action space and a reward function are designed, an Actor network and a Critic network are constructed based on a near-end strategy optimization algorithm, iterative training is carried out on the networks until convergence is carried out, and the convergence of a vehicle ramp entrance is realized. And finally, interacting with the SUMO through a TraCI interface to complete a ramp entrance confluence behavior. Other reward standards such as efficiency and comfort are added, so that the passing efficiency of the bottleneck road section can be effectively improved; the PPO deep reinforcement learning method based on the action-critic architecture can effectively solve the problem that when an existing deep reinforcement learning method processes a complex state space, the existing deep reinforcement learning method is prone to falling into the dilemma of state space explosion, and dimensionality disasters are caused. The method has the advantages that vehicle ramp confluence control can be achieved, the safety of vehicle driving in a ramp confluence area and the road traffic passing efficiency are improved, and meanwhile comfort of passengers is guaranteed.
一种基于深度强化学习的车辆匝道入口合流控制方法,其特征在于利用SUMO构建匝道合流仿真环境并获取相关状态信息,设计状态空间、动作空间以及奖励函数,基于近端策略优化算法构建Actor网络和Critic网络,并对网络进行迭代训练,直至收敛,最后通过TraCI接口与SUMO交互,完成匝道入口合流行为;加入了效率、舒适度等其他奖励标准,可以有效提升瓶颈路段的通行效率;采用actor‑critic架构的PPO深度强化学习方法,能够有效的解决现有深度强化学习方法在处理复杂状态空间时,容易陷入状态空间爆炸的困境,引发维度灾难问题。本发明具有可实现车辆匝道合流控制、提高了匝道合流区域车辆行驶的安全性及道路交通通行效率、同时保证乘客舒适的优点。
Vehicle ramp entrance confluence control method based on deep reinforcement learning
基于深度强化学习的车辆匝道入口合流控制方法
2023-06-06
Patent
Electronic Resource
Chinese
IPC: | B60W CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION , Gemeinsame Steuerung oder Regelung von Fahrzeug-Unteraggregaten verschiedenen Typs oder verschiedener Funktion / G06F ELECTRIC DIGITAL DATA PROCESSING , Elektrische digitale Datenverarbeitung / G08G Anlagen zur Steuerung, Regelung oder Überwachung des Verkehrs , TRAFFIC CONTROL SYSTEMS |
Entrance ramp cooperative confluence control method for intelligent network connection vehicles
European Patent Office | 2022
|European Patent Office | 2020
|Expressway entrance ramp confluence area double-layer optimization control method and system
European Patent Office | 2023
|Ramp confluence control method and ramp confluence control system for urban expressway
European Patent Office | 2020
|