The invention provides a passenger delay minimization signal control method based on deep reinforcement learning, and the method comprises the steps: determining each signal phase and a non-conflict phase, and calculating the saturation flow; collecting real-time vehicle data of all networked vehicles; calculating the time required for each phase vehicle to pass through the stop line under the continuous green light condition in the initial stage; defining a state, an action and an award of deep reinforcement learning based on an index of passenger delay, and merging and compressing a vehicle state and a signal state through a matrix; a pre-training and dual DQN combined deep reinforcement learning training method is adopted, a value function corresponding to a state and an action is estimated, a main network is adopted to select an action strategy, a target network is adopted to evaluate the value function, and a dual DQN method with regular updating is adopted to avoid falling into local optimum, and a phase scheme capable of obtaining the maximum passenger income is found. According to the invention, flexible signal phase optimization is adaptively carried out in combination with real-time vehicle data collected in real time, so that the average passenger delay at the intersection reaches the minimum.

    本发明提出了一种基于深度强化学习的乘客延误最小化信号控制方法,步骤为:确定各信号相位及其非冲突相位,计算饱和流量;收集所有联网车的实时车辆数据;计算初始阶段各相位车辆在持续绿灯条件下通过停车线所需时间;基于乘客延误的指标定义深度强化学习的状态、动作和奖励,将车辆状态和信号状态通过矩阵合并和压缩;采用预训练与双重DQN结合的深度强化学习训练方法,估算状态和动作对应的价值函数,并采用主网络选取动作策略、目标网络评估价值函数、定期更新的双重DQN方法避免陷入局部最优,找到能获得最大乘客收益的相位方案。本发明结合实时收集的实时车辆数据自适应地进行灵活的信号相位优化,使交叉口的平均乘客延误达到最低。


    Access

    Download


    Export, share and cite



    Title :

    Passenger delay minimization signal control method based on deep reinforcement learning


    Additional title:

    一种基于深度强化学习的乘客延误最小化信号控制方法


    Contributors:
    WU ZONGYUAN (author) / LI SHIMING (author) / DING SHENZHEN (author) / YIN CHUN'E (author)

    Publication date :

    2023-11-24


    Type of media :

    Patent


    Type of material :

    Electronic Resource


    Language :

    Chinese


    Classification :

    IPC:    G08G Anlagen zur Steuerung, Regelung oder Überwachung des Verkehrs , TRAFFIC CONTROL SYSTEMS / G06N COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS , Rechnersysteme, basierend auf spezifischen Rechenmodellen



    Deep Reinforcement Learning-Based Traffic Signal Control

    Hu, Penghui / Zhang, Xinran / Hu, Jianming | ASCE | 2024


    Deep Reinforcement Learning-based Traffic Signal Control

    Ruan, Junyun / Tang, Jinzhuo / Gao, Ge et al. | IEEE | 2023


    Traffic signal control method based on deep reinforcement learning

    LIU DUANYANG / SHEN SI / SHEN GUOJIANG et al. | European Patent Office | 2021

    Free access

    Urban intersection signal control method based on deep reinforcement learning

    JU YANNI / MA DIE | European Patent Office | 2024

    Free access

    Signal lamp intelligent control method based on deep reinforcement learning

    WEI KAI / ZHU YONG / ZHANG DONGHAI et al. | European Patent Office | 2024

    Free access