Highlights A multi-agent deep reinforcement learning framework is proposed for bus holding control. A reward function is defined to achieve headway self-equalization. The action of each agent is considered by introducing a joint-action tracker. A scheme based on proximal policy optimization is designed to train the agents. The framework outperforms other baselines in simulation studies.

    Abstract Bus bunching has been a long-standing problem that undermines the efficiency and reliability of public transport services. The most popular countermeasure in practice is to introduce static and dynamic holding control. However, most previous holding control strategies mainly consider local information with a pre-specified headway/schedule, while the global coordination of the whole bus fleet and its long-term effect are often overlooked. To efficiently incorporate global coordination and long-term operation in bus holding, in this paper we propose a multi-agent deep reinforcement learning (MDRL) framework to develop dynamic and flexible holding control strategies for a bus route. Specifically, we model each bus as an agent that interacts with not only its leader/follower but also all other vehicles in the fleet. To better explore potential strategies, we develop an effective headway-based reward function in the proposed framework. In the learning framework, we model fleet coordination by using a basic actor-critic scheme along with a joint action tracker to better characterize the complex interactions among agents in policy learning, and we apply proximal policy optimization to improve learning performance. We conduct extensive numerical experiments to evaluate the proposed MDRL framework against multiple baseline models that only rely on local information. Our results demonstrate the superiority of the proposed framework and show the promise of applying MDRL in the coordinative control of public transport vehicle fleets in real-world operations.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    Dynamic holding control to avoid bus bunching: A multi-agent deep reinforcement learning framework


    Beteiligte:
    Wang, Jiawei (Autor:in) / Sun, Lijun (Autor:in)


    Erscheinungsdatum :

    2020-04-11




    Medientyp :

    Aufsatz (Zeitschrift)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch




    Mitigating Bus Bunching via Hierarchical Multi-Agent Reinforcement Learning

    Yu, Mengdi / Yang, Tao / Li, Chunxiao et al. | IEEE | 2024




    Dynamic Holding Strategy to Prevent Buses from Bunching

    Ji, Yuxiong / Zhang, H. Michael | Transportation Research Record | 2013


    A Multi-agent Deep Reinforcement Learning Framework for UAV Swarm

    Zeng, Fanyu / Yang, Haigen / Zhao, Qian et al. | Springer Verlag | 2024