Reinforcement learning is an efficient, widely used machine learning technique that performs well in problems with a reasonable number of states and actions. This is rarely the case regarding control-related problems, as for instance controlling traffic signals, where the state space can be very large. One way to deal with the curse of dimensionality is to use generalization techniques such as function approximation. In this paper, a linear function approximation is used by traffic signal agents in a network of signalized intersections. Specifically, a true online SARSA $(\lambda)$ algorithm with Fourier basis functions (TOS( $\lambda $ )-FB) is employed. This method has the advantage of having convergence guarantees and error bounds, a drawback of non-linear function approximation. In order to evaluate TOS( $\lambda $ )-FB, we perform experiments in variations of an isolated intersection scenario and a scenario of the city of Cottbus, Germany, with 22 signalized intersections, implemented in MATSim. We compare our results not only to fixed-time controllers, but also to a state-of-the-art rule-based adaptive method, showing that TOS( $\lambda $ )-FB shows a performance that is highly superior to the fixed-time, while also being at least as efficient as the rule-based approach. For more than half of the intersections, our approach leads to less congestion and delay, without the need for the knowledge that underlies the rule-based approach.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Using Reinforcement Learning to Control Traffic Signals in a Real-World Scenario: An Approach Based on Linear Function Approximation


    Contributors:


    Publication date :

    2022-07-01


    Size :

    2247851 byte




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English



    A Reinforcement Learning Approach with Fourier Basis Linear Function Approximation for Traffic Signal Control

    Ziemke, Theresa / Alegre, Lucas N. / Bazzan, Ana L. C. | TIBKAT | 2020

    Free access



    Agent-Based Traffic Signal Control Using a Reinforcement Learning Approach

    Agafonov, Anton / Yumaganov, Alexander | IEEE | 2021


    Self-Organized Control of Traffic Signals Through Genetic Reinforcement Learning

    Mikami, S. / Kakazu, Y. / IEEE | British Library Conference Proceedings | 1993