The invention relates to a traffic signal lamp timing method, in particular to a traffic signal lamp timing method based on combination of deep reinforcement learning and extended Kalman filtering. The method comprises the following steps that: a traffic signal lamp timing system is abstracted as an intelligent agent, a traffic environment of a crossroad is abstracted as a controlled object, a timing scheme of a traffic signal lamp is abstracted as an action, the change of the accumulated waiting time of the vehicle is abstracted as a reward, and the intelligent agent selects an action according to the state information provided by a controlled object; after the action is executed, the controlled object feeds back a current state and the reward to the intelligent agent, the process is continuously repeated, and the intelligent agent continuously updates parameters by taking the maximum reward value as the target until an optimal action is obtained; and an experience pool is used for storing training samples and is used for DQN network training, an optimal estimation value of real parameters is obtained through continuous iteration updating of an EKF model, then the optimal estimation value of the real parameters is used for replacing an uncertainty parameter value to obtain an accurate value function, and an optimal timing scheme is selected.

    本发明涉及一种交通信号灯配时方法,尤其为一种基于深度强化学习与扩展卡尔曼滤波相结合的交通信号灯配时方法,包括将交通信号灯配时系统抽象为智能体,十字路口的交通环境抽象为被控对象,交通信号灯的配时方案抽象为动作,车辆的累计等待时间的变化抽象为奖励,智能体根据被控对象提供的状态信息选择动作;执行动作后,被控对象将当前状态和奖励反馈给智能体,并且不断重复此过程,智能体以获取最大奖励值为目标不断更新参数直至得到最优动作;经验池用来存储训练样本,并用于DQN网络训练,通过EKF模型的不断迭代更新得到真实参数的最优估计值,然后利用真实参数的最优估计值替代不确定性参数值得到准确的值函数并选择最优的配时方案。


    Access

    Download


    Export, share and cite



    Title :

    Traffic signal lamp timing method based on combination of deep reinforcement learning and extended Kalman filtering


    Additional title:

    基于深度强化学习与扩展卡尔曼滤波相结合的交通信号灯配时方法


    Contributors:
    WU LAN (author)

    Publication date :

    2022-01-18


    Type of media :

    Patent


    Type of material :

    Electronic Resource


    Language :

    Chinese


    Classification :

    IPC:    G08G Anlagen zur Steuerung, Regelung oder Überwachung des Verkehrs , TRAFFIC CONTROL SYSTEMS / G06N COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS , Rechnersysteme, basierend auf spezifischen Rechenmodellen



    Traffic signal lamp timing method, device and equipment based on deep reinforcement learning

    ZHOU JIANHONG / HUANG YUMIN / WANG YUNXIANG et al. | European Patent Office | 2023

    Free access

    Traffic signal lamp timing regulation and control method and system based on deep reinforcement learning

    GUO LIQUAN / XING HONGYE / WANG BO et al. | European Patent Office | 2024

    Free access

    Traffic signal lamp control system and method based on reinforcement learning and dynamic timing

    LIU BINGYI / SONG BIN / WU PENGCHUN et al. | European Patent Office | 2021

    Free access

    Traffic signal lamp timing method

    LI YING | European Patent Office | 2020

    Free access

    Traffic Signal Timing via Parallel Reinforcement Learning

    Zhao, Qian / Xu, Cheng / jin, Sheng | Springer Verlag | 2019