Abstract This paper proposes a physics-informed reinforcement learning(RL)-based ramp metering strategy, which trains the RL model using a combination of historic data and synthetic data generated from a traffic flow model. The optimal policy of the RL model is updated through an iterative training process, where in each iteration a new batch of historic data is collected and fed into the training data set. Such iterative training process can evaluate the control policy from reality rather than from a simulator, thus avoiding the RL model being trapped in an inaccurate training environment. The proposed strategy is applied to both local and coordinated ramp metering. Results from extensive microscopic simulation experiments demonstrate that the proposed strategy (i) significantly improves the traffic performance in terms of total time spent savings; (ii) outperforms classical feedback-based ramp metering strategies; and (iii) achieves higher improvements than an existing RL-based ramp metering strategy, which trains the RL model merely by a simulator. We also test the performance of two different learning algorithms in the simulation experiment, namely a conventional tabular approach and a batch-constrained deep RL approach. It is found that the deep RL approach is not as effective as the conventional tabular approach in the proposed strategy due to the limited amount of training data.

    Highlights Develop a physics-informed ramp control method that combines RL and a traffic flow model. Apply the proposed strategy to both local and coordinated ramp metering. Overcome the model mismatch problem in RL-based traffic control strategies. The proposed strategy outperforms well-known benchmark strategies in simulations.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    A physics-informed reinforcement learning-based strategy for local and coordinated ramp metering


    Contributors:
    Han, Yu (author) / Wang, Meng (author) / Li, Linghui (author) / Roncoli, Claudio (author) / Gao, Jinda (author) / Liu, Pan (author)


    Publication date :

    2022-01-24




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English





    Coordinated Ramp Metering with Equity Consideration Using Reinforcement Learning

    Lu, Chao / Huang, Jie / Deng, Lianbo et al. | ASCE | 2017



    A hierarchy control strategy for coordinated ramp metering

    Huifang Wang, / Yangzhou Chen, / Zhigang Shi, | IEEE | 2003


    A Hierarchy Control Strategy for Coordinated Ramp Metering

    Wang, H. / Chen, Y. / Shi, Z. et al. | British Library Conference Proceedings | 2003