The invention relates to a traffic signal lamp timing method, in particular to a traffic signal lamp timing method based on combination of deep reinforcement learning and extended Kalman filtering. The method comprises the following steps that: a traffic signal lamp timing system is abstracted as an intelligent agent, a traffic environment of a crossroad is abstracted as a controlled object, a timing scheme of a traffic signal lamp is abstracted as an action, the change of the accumulated waiting time of the vehicle is abstracted as a reward, and the intelligent agent selects an action according to the state information provided by a controlled object; after the action is executed, the controlled object feeds back a current state and the reward to the intelligent agent, the process is continuously repeated, and the intelligent agent continuously updates parameters by taking the maximum reward value as the target until an optimal action is obtained; and an experience pool is used for storing training samples and is used for DQN network training, an optimal estimation value of real parameters is obtained through continuous iteration updating of an EKF model, then the optimal estimation value of the real parameters is used for replacing an uncertainty parameter value to obtain an accurate value function, and an optimal timing scheme is selected.
本发明涉及一种交通信号灯配时方法,尤其为一种基于深度强化学习与扩展卡尔曼滤波相结合的交通信号灯配时方法,包括将交通信号灯配时系统抽象为智能体,十字路口的交通环境抽象为被控对象,交通信号灯的配时方案抽象为动作,车辆的累计等待时间的变化抽象为奖励,智能体根据被控对象提供的状态信息选择动作;执行动作后,被控对象将当前状态和奖励反馈给智能体,并且不断重复此过程,智能体以获取最大奖励值为目标不断更新参数直至得到最优动作;经验池用来存储训练样本,并用于DQN网络训练,通过EKF模型的不断迭代更新得到真实参数的最优估计值,然后利用真实参数的最优估计值替代不确定性参数值得到准确的值函数并选择最优的配时方案。
Traffic signal lamp timing method based on combination of deep reinforcement learning and extended Kalman filtering
基于深度强化学习与扩展卡尔曼滤波相结合的交通信号灯配时方法
2022-01-18
Patent
Electronic Resource
Chinese
Traffic signal lamp timing method, device and equipment based on deep reinforcement learning
European Patent Office | 2023
|European Patent Office | 2024
|Traffic signal lamp control system and method based on reinforcement learning and dynamic timing
European Patent Office | 2021
|Traffic Signal Timing via Parallel Reinforcement Learning
Springer Verlag | 2019
|