The invention discloses a ramp coordination control method based on different strategy reinforcement learning, which uses off-line traffic data to train a reinforcement learning agent in an iteration mode instead of interacting with a traffic simulator. In each training iteration, historical traffic flow data is collected and input into a learning algorithm to update a control strategy. In the initial stage of iteration, when the historical traffic flow data amount is small, a macroscopic traffic flow model METANET is used to generate synthetic traffic flow data. This enables reinforcement learning to explore new actions that may result in better traffic performance. In order to ensure the feasible time of training iteration, the expressway is divided into a plurality of single-point control areas, and each single-point control area corresponds to a single-point ramp control system to maintain a low-dimensional state and action space, so that control actions can be fully explored. Along with exploration of more control actions and utilization of more training data, an optimal strategy of ramp coordination control is finally obtained.

    本发明公开了一种基于异策略强化学习的匝道协调控制方法,以迭代方式使用离线交通数据训练强化学习智能体,而不是与交通模拟器交互。在每次训练迭代中,收集历史交通流数据并将其输入学习算法以更新控制策略。在迭代初期,当历史交通流数据量较少时,使用宏观交通流模型METANET来生成合成交通流数据。这使强化学习能够探索到可能导致更好的交通性能的新动作。为了保证训练迭代的可行时间,快速道路被分为若干单点控制区域,每个单点控制区域对应于单点匝道控制系统,以保持低维度的状态和动作空间,从而可以充分探索控制动作。随着探索更多的控制动作和利用更多的训练数据,最终得到匝道协调控制的最优策略。


    Zugriff

    Download


    Exportieren, teilen und zitieren



    Titel :

    Multi-ramp coordination control method based on different strategy reinforcement learning


    Weitere Titelangaben:

    一种基于异策略强化学习的多匝道协调控制方法


    Beteiligte:
    HAN YU (Autor:in) / GAO JINDA (Autor:in) / LI LINGHUI (Autor:in) / LIU PAN (Autor:in)

    Erscheinungsdatum :

    2022-04-08


    Medientyp :

    Patent


    Format :

    Elektronische Ressource


    Sprache :

    Chinesisch


    Klassifikation :

    IPC:    G08G Anlagen zur Steuerung, Regelung oder Überwachung des Verkehrs , TRAFFIC CONTROL SYSTEMS / G06N COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS , Rechnersysteme, basierend auf spezifischen Rechenmodellen / G06Q Datenverarbeitungssysteme oder -verfahren, besonders angepasst an verwaltungstechnische, geschäftliche, finanzielle oder betriebswirtschaftliche Zwecke, sowie an geschäftsbezogene Überwachungs- oder Voraussagezwecke , DATA PROCESSING SYSTEMS OR METHODS, SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL, SUPERVISORY OR FORECASTING PURPOSES



    A Multi-Hierarchical Strategy for On-Ramp Coordination

    Jiang, Rui / Lee, Jinwoo (Brian) / Chung, Edward | Springer Verlag | 2015


    Mixed multi-ramp cooperative confluence control method based on multi-agent reinforcement learning

    YAN JUNQI / YU HONGXIN / ZHANG LIHUI et al. | Europäisches Patentamt | 2023

    Freier Zugriff

    GCS: Graph-Based Coordination Strategy for Multi-Agent Reinforcement Learning

    Ruan, J / Du, Y / Xiong, X et al. | BASE | 2022

    Freier Zugriff

    Ramp control method based on offline reinforcement learning and macroscopic model

    HAN YU / LI LINGHUI / GAO JINDA et al. | Europäisches Patentamt | 2022

    Freier Zugriff

    Ramp signal control optimization method and system based on reinforcement learning

    JI SHENGYI / WANG XIANG / WANG XI et al. | Europäisches Patentamt | 2021

    Freier Zugriff