Autonomous Intersection Management (AIM) systems present a new paradigm for conflict-free cooperation of connected autonomous vehicles (CAVs) at road intersections, the aim of which is to eliminate collisions and improve the traffic efficiency and ride comfort. Given the challenges of current centralized coordination methods in balancing high computational efficiency and robust safety assurance, this paper proposes an innovative conflict-free management scheme for CAVs at unsignalized intersections, leveraging safe multi-agent deep reinforcement learning (MADRL). Firstly, we formulate the safe MADRL problem as a constrained Markov game (CMG) and then transform the AIM problem into a CMG by carefully designing state, action, reward, and cost functions. Subsequently, we propose the Multi-Agent Constrained Policy Optimization (MACPO), specifically tailored to solve the CMG problem. MACPO incorporates safety constraints that further restrict the trust region formed by the Kullback-Leibler (KL) divergence, facilitating reinforcement learning policy updates that maximize performance while keeping constraint costs within their limit bounds. This leads us to introduce the MACPO-based AIM Algorithm. Finally, we train an AIM policy and compare its computation time, ride comfort, traffic efficiency, and safety with management schemes based on Model Predictive Control (MPC), Mixed Integer Programming (MIP), and non-safety-aware reinforcement learning. According to the results, compared with the MPC and MIP methods, our method has increased computational efficiency by 65.22 times and 731.52 times respectively, and has improved traffic efficiency by 2.41 times and 1.80 times respectively. In contrast to the non-safety awareness RL methods, our method achieves a zero collision rate for the first time, while also enhancing ride comfort, highlighting the advantages of using MACPO.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    Multi-Agent Constrained Policy Optimization for Conflict-Free Management of Connected Autonomous Vehicles at Unsignalized Intersections


    Beteiligte:
    Zhao, Rui (Autor:in) / Li, Yun (Autor:in) / Gao, Fei (Autor:in) / Gao, Zhenhai (Autor:in) / Zhang, Tianyao (Autor:in)


    Erscheinungsdatum :

    01.06.2024


    Format / Umfang :

    2055617 byte




    Medientyp :

    Aufsatz (Zeitschrift)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch



    Connected autonomous vehicle driving path optimization method for unsignalized intersections

    Zhang, Zhi / Rao, Bowen / Long, Guansen et al. | British Library Conference Proceedings | 2023




    On the Awareness of Connected Vehicles at Unsignalized Intersections

    Avedisov, Sergei S. / Higuchi, Takamasa / Sakr, Ahmed Hamdi et al. | IEEE | 2022


    Can Autonomous Vehicles Enhance Traffic Safety at Unsignalized Intersections?

    Maryam Mousavi, Seyedeh / Lord, Dominique / Dadashova, Bahar et al. | ASCE | 2020