The invention discloses a self-adaptive traffic signal control method based on CQL offline reinforcement learning, relates to the technical field of traffic signal control, and realizes reinforcement learning by learning a value function. The value function represents long-term return expected values of different actions in the current state; in other words, only existing historical data is used for training; according to the method, offline learning is realized by learning the difference between a target strategy and a behavior strategy; wherein the target strategy is a finally required optimal strategy, and the behavior strategy is a strategy adopted in historical data. In the algorithm, the optimal strategy is learned by maximizing the maximum long-term return in each state. According to the algorithm, a new constraint condition is added, so that the target strategy is in the distribution range of the behavior strategy. The constraint condition can ensure that the learned target strategy is well expressed in the state and action of historical data, and meanwhile, the overfitting problem occurring in the learning process is also avoided.

    本发明公开了一种基于CQL离线强化学习的自适应交通信号控制方法,涉及交通信号控制技术领域,通过学习一个价值函数来实现强化学习;该价值函数表示在当前状态下,采取不同动作的长期回报期望值;即只使用已有的历史数据进行训练;它通过学习一个目标策略和一个行为策略的差异来实现离线学习;其中,目标策略是最终要求的最优策略,行为策略是在历史数据中采取的策略。在本算法中,通过最大化每个状态下的最大长期回报来学习最优策略。本算法加入了一个新的约束条件,使得目标策略在行为策略的分布范围内。这个约束条件能够保证学习出的目标策略在历史数据中出现的状态和动作上表现良好,同时也避免了学习过程中出现的过拟合问题。


    Access

    Download


    Export, share and cite



    Title :

    Adaptive traffic signal control method based on CQL offline reinforcement learning


    Additional title:

    一种基于CQL离线强化学习的自适应交通信号控制方法


    Contributors:
    PI JIATIAN (author) / YANG XINMIN (author) / WU CHANGZHI (author)

    Publication date :

    2023-11-03


    Type of media :

    Patent


    Type of material :

    Electronic Resource


    Language :

    Chinese


    Classification :

    IPC:    G08G Anlagen zur Steuerung, Regelung oder Überwachung des Verkehrs , TRAFFIC CONTROL SYSTEMS / G06F ELECTRIC DIGITAL DATA PROCESSING , Elektrische digitale Datenverarbeitung / G06N COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS , Rechnersysteme, basierend auf spezifischen Rechenmodellen




    Reinforcement Learning for True Adaptive Traffic Signal Control

    Abdulhai, B. / Pringle, R. / Karakoulas, G. J. | British Library Online Contents | 2003


    Adaptive traffic signal control method based on multi-agent reinforcement learning

    ZHANG CHENGWEI / JIN SHAN / ZHENG KANGJIE | European Patent Office | 2021

    Free access

    Adaptive traffic signal control method combining traffic flow prediction and reinforcement learning

    PI JIATIAN / YANG XINMIN / WU CHANGZHI | European Patent Office | 2023

    Free access

    Signal lamp and route cooperative control method based on offline reinforcement learning

    OUYANG YAJIE / ZHAO KUO | European Patent Office | 2024

    Free access