Highlights A multi-constraint model for impulsive orbital pursuit-evasion game (OPEG) is constructed. A learning-based algorithm (i.e. PRD-MADDPG) for impulsive OPEG is designed. The Predict-Reward-Detect (PRD) training framework for improving MADDPG is established. The PRD-MADDPG performs better than conventional MADDPG in impulsive OPEGs. The winning mechanism of the game for both players under multiple factors is analyzed.

    Abstract This paper comprehensively investigates the problem of impulsive orbital pursuit-evasion games (OPEGs) by using an artificial intelligence-based approach. First, the mathematical model for the impulsive OPEGs in which the pursuer and evader both perform their orbital maneuvers by imposing the impulsive velocity increments is constructed. Second, the problem of impulsive OPEGs is transformed into a bilateral optimization problem with a minimum–maximum optimization index in terms of terminal time and multiple constraints such as maneuverability, total fuel consumption, and mission time, etc. To determine the optimal impulsive maneuvers for both sides, a PRD-MADDPG (Predict-Reward-Detect Multi-Agent Deep Deterministic Policy Gradient) algorithm in the frame of multi-agent reinforcement learning is designed. This novel algorithm uses the basic MADDPG to achieve the strategies training and learning, and applies the supplemental PRD to predict the change of game state during the interval between two adjacent impulsive maneuvers and incorporate these information into the algorithm training in the form of predicted reward. Finally, some pursuit-evasion missions near the Geosynchronous Earth Orbit are numerically analyzed to verify the validness and effectiveness of the algorithm. The results prove that the PRD-MADDPG algorithm is very efficient to find applicable strategies even considering rather complex constraints. It is also shown that the learning-based strategies can be effectively applied in the extended scenarios which are not seen in the training process.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    PRD-MADDPG: An efficient learning-based algorithm for orbital pursuit-evasion game with impulsive maneuvers


    Contributors:
    Zhao, Liran (author) / Zhang, Yulin (author) / Dang, Zhaohui (author)

    Published in:

    Advances in Space Research ; 72 , 2 ; 211-230


    Publication date :

    2023-03-05


    Size :

    20 pages




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English




    Pulse type track pursuit game method based on PRD-MADDPG algorithm

    ZHAO LIRAN / DANG CHAOHUI / TANG SHENGYONG et al. | European Patent Office | 2022

    Free access

    Revisit of the Three-Dimensional Orbital Pursuit-Evasion Game

    Shen, Hong-Xin / Casalino, Lorenzo | AIAA | 2018


    Pursuit-evasion orbital game for satellite interception and collision avoidance

    Shen, Dan / Pham, Khanh / Blasch, Erik et al. | SPIE | 2011


    Orbital Pursuit-Evasion Hybrid Spacecraft Controllers

    Hafer, William T. / Reed, Helen L. | AIAA | 2015