Adaptive Traffic Signal Control methods based on Reinforcement Learning have been applied successfully to reach state-of-the-art results in simulation. However, most recent works in the area use model-free methods, which learn value and/or policy functions directly through environment in-teractions without a dynamics model. Model-free methods have several potential drawbacks: (1) difficulties in generalization to dynamics not seen in training, as well as difficulties in transfer to different intersection geometries; (2) low sample-efficiency, resulting in high resource requirements for training; (3) often, reliance on difficult-to-interpret function approximators such as neural networks. In this work, we demonstrate experimentally that Monte Carlo Tree Search (MCTS) with a simple queue and platoon-propagation dynamics model can reach lower delay on a single intersection than the standard model-free Deep Q-Networks (DQN) algorithm. Moreover, we provide empirical results that illustrate that model-based receding-horizon planning can alleviate many of the drawbacks of the model-free methods listed above: better generalization characteristics; more direct and less resource-demanding transfer to different intersection geometries and phasing schemes; and simpler visualization and interpretation, aiding modification and debugging.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    A Case for Monte Carlo Tree Search in Adaptive Traffic Signal Control: Modifiability, Interpretability and Generalization


    Beteiligte:
    Smirnov, Ilia (Autor:in) / Sanner, Scott (Autor:in) / Abdulhai, Baher (Autor:in)


    Erscheinungsdatum :

    24.09.2023


    Format / Umfang :

    919827 byte





    Medientyp :

    Aufsatz (Konferenz)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch



    Monte Carlo Tree Search-Based Mixed Traffic Flow Control Algorithm for Arterial Intersections

    Cheng, Yanqiu / Hu, Xianbiao / Tang, Qing et al. | Transportation Research Record | 2020


    Adaptive Reward for CAV Action Planning using Monte Carlo Tree Search

    Patel, Dhruvkumar / Zalila-Wenkstern, Rym | IEEE | 2021


    Multi-Objective Monte-Carlo Tree Search based aerial maneuvering control

    Xiang Chen / Xinguo Zhang / Weijia Wang et al. | IEEE | 2016


    MONTE CARLO TREE SEARCH METHODS FOR TELESCOPE TASKING

    Fedeler, Samuel / Holzinger, Marcus | TIBKAT | 2020


    Monte Carlo Tree Search Methods for Telescope Tasking

    Fedeler, Samuel / Holzinger, Marcus | AIAA | 2020