The invention discloses a traffic light signal control method based on an Actor-Critic framework deep reinforcement learning algorithm, and the method comprises the steps: building a simulation environment, defining the state, action and reward value of a deep reinforcement learning model, and setting and initializing a strategy network, an old strategy network and a value function network; collecting intersection traffic state information, and generating a timing scheme of a next signal period based on an old strategy network; collecting training data to carry out generalized advantage estimation, calculating training errors of a strategy network and a value function network, and updating network parameters; copying the strategy network parameters to an old strategy network; and utilizing the trained signal control model to generate a signal timing scheme of the next period based on the traffic state information of the intersection at the current moment. According to the method, the model training efficiency of the signal control model can be remarkably improved, the indexes such as the average queuing length of the intersection, the average travel time and the average delay of vehicles are remarkably reduced, and the generated signal timing scheme has higher practicability and safety.

    基于Actor‑Critic框架深度强化学习算法的交通灯信号控制方法,包括:建立仿真环境,定义深度强化学习模型的状态、动作、奖励值,设定并初始化策略网络、旧策略网络和值函数网络;采集交叉口交通状态信息,基于旧策略网络生成下一信号周期的配时方案;采集训练数据进行广义优势估计,计算策略网络和值函数网络的训练误差,并更新网络参数;将策略网络参数复制给旧策略网络;利用训练好的信号控制模型,基于当前时刻交叉口的交通状态信息,生成下一周期的信号配时方案。本发明能够显著提升信号控制模型的模型训练效率,明显降低交叉口平均排队长度、平均旅行时间和车辆平均延误等指标,且本发明生成的信号配时方案具备更高的实用性和安全性。


    Access

    Download


    Export, share and cite



    Title :

    Traffic light signal control method based on Actor-Critic framework deep reinforcement learning algorithm


    Additional title:

    基于Actor-Critic框架深度强化学习算法的交通灯信号控制方法


    Contributors:
    SHEN GUOJIANG (author) / SHEN SI (author) / KONG XIANGJIE (author) / ZHENG JIANWEI (author) / ZHANG MEIYU (author)

    Publication date :

    2021-04-09


    Type of media :

    Patent


    Type of material :

    Electronic Resource


    Language :

    Chinese


    Classification :

    IPC:    G06F ELECTRIC DIGITAL DATA PROCESSING , Elektrische digitale Datenverarbeitung / G06N COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS , Rechnersysteme, basierend auf spezifischen Rechenmodellen / G08G Anlagen zur Steuerung, Regelung oder Überwachung des Verkehrs , TRAFFIC CONTROL SYSTEMS





    Soft Actor-Critic Deep Reinforcement Learning for Fault Tolerant Flight Control

    Dally, Killian / Kampen, Erik-Jan Van | TIBKAT | 2022


    Actor-Critic Reinforcement Learning for Control With Stability Guarantee

    Han, M / Zhang, L / Wang, J et al. | BASE | 2020

    Free access

    Multiagent Soft Actor–Critic for Traffic Light Timing

    Wu, Lan / Wu, Yuanming / Qiao, Cong et al. | ASCE | 2023