Abstract Reinforcement learning (RL) techniques have been studied for solving the conflict resolution (CR) problem in air traffic management, leveraging their potential for computation and ability to handle uncertainty. However, challenges remain that impede the application of RL methods to CR in practice, including three-dimensional manoeuvres, generalisation, trajectory recovery, and success rate. This paper proposes a general multi-agent reinforcement learning approach for real-time three-dimensional multi-aircraft conflict resolution, in which agents share a neural network and are deployed on each aircraft to form a distributed decision-making system. To address the challenges, several technologies are introduced, including a partial observation model based on imminent threats for generalisation, a safety separation relaxation model for multiple flight levels for three-dimensional manoeuvres, an adaptive manoeuvre strategy for trajectory recovery, and a conflict buffer model for success rate. The Rainbow Deep Q-learning Network (DQN) is used to enhance the efficiency of the RL process. A simulation environment that considers flight uncertainty (resulting from mechanical and navigation errors and wind) is constructed to train and evaluate the proposed approach. The experimental results demonstrate that the proposed method can resolve conflicts in scenarios with much higher traffic density than in today’s real-world situations.

    Highlights The proposed method faces real-time 3D high-density scenarios with uncertainty. The proposed method is compatible with the flight-level-based operation mode. A safety separation relaxation model simplifies 3D conflict identification. The relationship between observation modules and performances was explored. The 2D and 3D MARL models were compared and analysed.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    General real-time three-dimensional multi-aircraft conflict resolution method using multi-agent reinforcement learning


    Beteiligte:
    Chen, Yutong (Autor:in) / Xu, Yan (Autor:in) / Yang, Lei (Autor:in) / Hu, Minghua (Autor:in)


    Erscheinungsdatum :

    2023-10-01




    Medientyp :

    Aufsatz (Zeitschrift)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch





    Graph Reinforcement Learning for Multi-Aircraft Conflict Resolution

    Li, Yumeng / Zhang, Yunhe / Guo, Tong et al. | IEEE | 2024




    Optimal coordinated maneuvers for three dimensional aircraft conflict resolution

    Hu, Jianghai / Prandini, Maria / Sastry, Shankar et al. | AIAA | 2001