Adaptive traffic signal control (ATSC) can ease the increasing congestion to relieve pressure on metropolitan transportation systems. In a large-scale road network, ATSC has a high dimensional action space, which makes training very slow and algorithms difficult to converge for conventional centralized deep reinforcement learning (DRL) approaches. Multi-agent reinforcement learning (MARL) approach overcomes this issue by decomposing the joint action space to several sub-spaces and each agent searches the optimal action in its own space. However, if all the agents make their decisions independently and only maximize their own reward, the state transition probability of the environment in a Markov decision process (MDP) will come to be unstable and ATSC will not converge to the optimal policy finally. To let agents learn to cooperate, this paper proposes a novel MARL method where a difference reward overcomes the credit assignment issue among cooperated agents. Moreover, a spatially weighted reward, which can let agents consider the reward of their neighbors in the road network, is designed to evaluate the policies of decentralized actor networks such that the cooperation among agents is reinforced. By comparing against the independent DRL approach and other multi-agent approaches in a large-scale network, our proposed MARL approach is demonstrated that its effectiveness in terms of average reward and travel delay is over other approaches.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    Cooperative Learning with Difference Reward in Large-Scale Traffic Signal Control


    Beteiligte:
    Wang, Shuo (Autor:in) / Yue, Wenwei (Autor:in) / Chen, Yue (Autor:in) / Ji, Xingyi (Autor:in) / Chen, Xinyang (Autor:in) / Jin, Yaokai (Autor:in)


    Erscheinungsdatum :

    2022-10-08


    Format / Umfang :

    801599 byte




    Medientyp :

    Aufsatz (Konferenz)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch




    Reinforcement learning traffic signal control method based on internal reward enhancement

    REN YILONG / FU XIANG / CHEN JIAXU et al. | Europäisches Patentamt | 2024

    Freier Zugriff

    Reward Function Design in Multi-Agent Reinforcement Learning for Traffic Signal Control

    Koohy, Behrad / Stein, Sebastian / Gerding, Enrico et al. | TIBKAT | 2022

    Freier Zugriff

    Adaptive traffic signal control system using composite reward architecture based deep reinforcement learning

    Jamil, Abu Rafe Md / Ganguly, Kishan Kumar / Nower, Naushin | Wiley | 2020

    Freier Zugriff

    Adaptive traffic signal control system using composite reward architecture based deep reinforcement learning

    Jamil, Abu Rafe Md / Ganguly, Kishan Kumar / Nower, Naushin | IET | 2021

    Freier Zugriff