With the development of computer science, automatic control, aircraft design and other disciplines, artificial intelligence-driven Unmanned Combat Aerial Vehicle (UCAV) air combat decision-making technology has brought revolutionary changes in air combat theory and mode. Aiming at the six-degree-of-freedom UCAV close-range air combat autonomous decision-making problem, this paper proposes a UCAV air combat decision-making method based on the deep reinforcement learning method. Firstly, a close-range air combat environment model based on the six-degree-of-freedom UCAV model is developed. Secondly, an autonomous decision-making model for the UCAV close-range air combat with multi-dimensional continuous state input and multi-dimensional continuous action output is established based on the deep neural network, which receives the combat situation information and outputs the UCAV’s joystick displacement commands. Then, a reward function considering the missile attack zone and air combat orientation is designed, which includes the angle reward, the distance reward and the height reward. On this basis, a twin delayed deep deterministic policy gradient algorithm is employed to train the autonomous decision-making model for air combat. Finally, simulation experiments of the UCAV close-range air combat scenario are carried out, and the simulation results show that the proposed intelligent air combat decision-making machine has a win rate 3.57 times higher than that of an expert system, and occupies an average situation reward 1.19 times higher than that of the enemy aircraft.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Deep Reinforcement Learning-Based Decision Making for Six Degree of Freedom UCAV Close Range Air Combat


    Additional title:

    Lect. Notes Electrical Eng.


    Contributors:
    Fu, Song (editor) / Zhou, Pan (author) / Li, Ni (author) / Huang, Jiangtao (author) / Zhang, Sheng (author) / Zhou, Xiaoyu (author) / Liu, Gang (author)

    Conference:

    Asia-Pacific International Symposium on Aerospace Technology ; 2023 ; Lingshui, China October 16, 2023 - October 18, 2023



    Publication date :

    2024-07-05


    Size :

    15 pages





    Type of media :

    Article/Chapter (Book)


    Type of material :

    Electronic Resource


    Language :

    English




    Decision-making and confrontation in close-range air combat based on reinforcement learning

    YANG, Mengchao / SHAN, Shengzhe / ZHANG, Weiwei | Elsevier | 2025

    Free access


    A UCAV Maneuver Decision-Making Framework for One-on-One Air Combat

    Xuan, Yongbo / Zhou, Ke / Wu, Bi et al. | British Library Conference Proceedings | 2022


    A UCAV Maneuver Decision-Making Framework for One-on-One Air Combat

    Xuan, Yongbo / Zhou, Ke / Wu, Bi et al. | Springer Verlag | 2022


    A UCAV Maneuver Decision-Making Framework for One-on-One Air Combat

    Xuan, Yongbo / Zhou, Ke / Wu, Bi et al. | TIBKAT | 2022