The invention discloses a ramp control method based on offline reinforcement learning and a macroscopic model, and the method comprises the steps: obtaining highway geometric parameters, road traffic data and control information, and building and calibrating a macroscopic traffic flow model; generating synthetic reinforcement learning data based on the macroscopic traffic flow model, and mixing the synthetic reinforcement learning data with historical data to obtain a reinforcement learning data set; offline reinforcement learning is carried out; a strategy is updated, online control is carried out, and road traffic data and control information are collected; and repeating the steps for iterative training. According to the method, system errors caused by inherent mismatching of a microscopic simulation platform and actual traffic conditions are avoided, the convergence problem of complete offline learning is also avoided, meanwhile, the efficiency of reinforcement learning action exploration is improved through the macroscopic traffic model, the problem of limited data in an actual environment is solved, and the method is suitable for popularization and application. According to the final scheme, traffic congestion is avoided to a great extent, and the traffic operation efficiency is improved.

    本发明公开了一种基于线下强化学习与宏观模型的匝道控制方法,包括:获取高速公路几何参数,道路交通数据以及控制信息,建立并标定宏观交通流模型;基于宏观交通流模型中生成合成强化学习数据,与历史数据混合获得强化学习数据集;进行线下强化学习:更新策略进行线上控制并采集道路交通数据以及控制信息;重复上述步骤迭代训练。本发明提出的既避免了通过微观仿真平台与实际交通条件的固有不匹配所带来的系统误差,也避免了完全线下学习的收敛问题,同时宏观交通模型提高了强化学习动作探索的效率,解决了实际环境中数据有限的问题,最终方案很大程度避免了交通拥堵,提高了交通运行的效率。


    Zugriff

    Download


    Exportieren, teilen und zitieren



    Titel :

    Ramp control method based on offline reinforcement learning and macroscopic model


    Weitere Titelangaben:

    基于线下强化学习与宏观模型的匝道控制方法


    Beteiligte:
    HAN YU (Autor:in) / LI LINGHUI (Autor:in) / GAO JINDA (Autor:in) / LIU PAN (Autor:in)

    Erscheinungsdatum :

    2022-03-04


    Medientyp :

    Patent


    Format :

    Elektronische Ressource


    Sprache :

    Chinesisch


    Klassifikation :

    IPC:    G08G Anlagen zur Steuerung, Regelung oder Überwachung des Verkehrs , TRAFFIC CONTROL SYSTEMS



    Ramp signal control optimization method and system based on reinforcement learning

    JI SHENGYI / WANG XIANG / WANG XI et al. | Europäisches Patentamt | 2021

    Freier Zugriff

    Adaptive traffic signal control method based on CQL offline reinforcement learning

    PI JIATIAN / YANG XINMIN / WU CHANGZHI | Europäisches Patentamt | 2023

    Freier Zugriff

    Multi-ramp coordination control method based on different strategy reinforcement learning

    HAN YU / GAO JINDA / LI LINGHUI et al. | Europäisches Patentamt | 2022

    Freier Zugriff

    Vehicle ramp entrance confluence control method based on deep reinforcement learning

    TONG NING / LI FENGQI / WANG KUNRUI | Europäisches Patentamt | 2023

    Freier Zugriff

    Reinforcement Learning for Ramp Control: An Analysis of Learning Parameters

    Chao Lu / Jie Huang / Jianwei Gong | DOAJ | 2016

    Freier Zugriff