Adaptive Traffic Signal Control methods based on Reinforcement Learning have been applied successfully to reach state-of-the-art results in simulation. However, most recent works in the area use model-free methods, which learn value and/or policy functions directly through environment in-teractions without a dynamics model. Model-free methods have several potential drawbacks: (1) difficulties in generalization to dynamics not seen in training, as well as difficulties in transfer to different intersection geometries; (2) low sample-efficiency, resulting in high resource requirements for training; (3) often, reliance on difficult-to-interpret function approximators such as neural networks. In this work, we demonstrate experimentally that Monte Carlo Tree Search (MCTS) with a simple queue and platoon-propagation dynamics model can reach lower delay on a single intersection than the standard model-free Deep Q-Networks (DQN) algorithm. Moreover, we provide empirical results that illustrate that model-based receding-horizon planning can alleviate many of the drawbacks of the model-free methods listed above: better generalization characteristics; more direct and less resource-demanding transfer to different intersection geometries and phasing schemes; and simpler visualization and interpretation, aiding modification and debugging.
A Case for Monte Carlo Tree Search in Adaptive Traffic Signal Control: Modifiability, Interpretability and Generalization
24.09.2023
919827 byte
Aufsatz (Konferenz)
Elektronische Ressource
Englisch
Monte Carlo Tree Search-Based Mixed Traffic Flow Control Algorithm for Arterial Intersections
Transportation Research Record | 2020
|MONTE CARLO TREE SEARCH METHODS FOR TELESCOPE TASKING
TIBKAT | 2020
|Monte Carlo Tree Search Methods for Telescope Tasking
AIAA | 2020
|