Transportation systems have a significant impact on carbon emissions and urban mobility worldwide. Previous work has shown that reinforcement-learning-based control of autonomous vehicles (AVs) can smooth traffic flow and improve congestion. Due to the safety and interpretability concerns around deploying AVs, we propose structured reinforcement learning policies to provide real-time guidance to human drivers to stabilize traffic. Motivated by the limitations of driver reaction times, we introduce a class of piecewise constant policies designed to be executed by human drivers to mitigate congestion. These policies are defined by a simple action space and an action extension parameter Δ where each action must be held constant for Δ timesteps. We prove that the optimal piecewise constant trajectory deviates in reward from the optimal trajectory by at most O(HΔ) over a horizon of length H. We show that piecewise constant policies can reproduce traffic smoothing in single-lane settings for large action extension parameters (up to 14s), yielding near-optimal average velocities of above 4.2 m/s compared to 3.6 m/s by a human driver baseline. Moreover, these policies see some improvements for action extension parameters up to 24s and are robust to lane changing, indicating promise for more complex environments.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Piecewise Constant Policies for Human-Compatible Congestion Mitigation


    Contributors:
    Sridhar, Mayuri (author) / Wu, Cathy (author)


    Publication date :

    2021-09-19


    Size :

    1178979 byte




    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English



    Cooperative Advisory Residual Policies for Congestion Mitigation

    Hasan, Aamir / Chakraborty, Neeloy / Chen, Haonan et al. | ArXiv | 2024

    Free access


    Multipath Routing Compatible Congestion Control

    Chang, Tianfang / Cai, Lin | IEEE | 2024


    LOCAL CONGESTION MITIGATION

    WILLIAMS RALPH MAXWELL / PALKOPOULOU ELENI | European Patent Office | 2022

    Free access

    PeRP: Personalized Residual Policies For Congestion Mitigation Through Co-operative Advisory Systems

    Hasan, Aamir / Chakraborty, Neeloy / Chen, Haonan et al. | ArXiv | 2023

    Free access