Highlights A multi-constraint model for impulsive orbital pursuit-evasion game (OPEG) is constructed. A learning-based algorithm (i.e. PRD-MADDPG) for impulsive OPEG is designed. The Predict-Reward-Detect (PRD) training framework for improving MADDPG is established. The PRD-MADDPG performs better than conventional MADDPG in impulsive OPEGs. The winning mechanism of the game for both players under multiple factors is analyzed.
Abstract This paper comprehensively investigates the problem of impulsive orbital pursuit-evasion games (OPEGs) by using an artificial intelligence-based approach. First, the mathematical model for the impulsive OPEGs in which the pursuer and evader both perform their orbital maneuvers by imposing the impulsive velocity increments is constructed. Second, the problem of impulsive OPEGs is transformed into a bilateral optimization problem with a minimum–maximum optimization index in terms of terminal time and multiple constraints such as maneuverability, total fuel consumption, and mission time, etc. To determine the optimal impulsive maneuvers for both sides, a PRD-MADDPG (Predict-Reward-Detect Multi-Agent Deep Deterministic Policy Gradient) algorithm in the frame of multi-agent reinforcement learning is designed. This novel algorithm uses the basic MADDPG to achieve the strategies training and learning, and applies the supplemental PRD to predict the change of game state during the interval between two adjacent impulsive maneuvers and incorporate these information into the algorithm training in the form of predicted reward. Finally, some pursuit-evasion missions near the Geosynchronous Earth Orbit are numerically analyzed to verify the validness and effectiveness of the algorithm. The results prove that the PRD-MADDPG algorithm is very efficient to find applicable strategies even considering rather complex constraints. It is also shown that the learning-based strategies can be effectively applied in the extended scenarios which are not seen in the training process.
PRD-MADDPG: An efficient learning-based algorithm for orbital pursuit-evasion game with impulsive maneuvers
Advances in Space Research ; 72 , 2 ; 211-230
2023-03-05
20 pages
Article (Journal)
Electronic Resource
English
Pulse type track pursuit game method based on PRD-MADDPG algorithm
European Patent Office | 2022
|Orbital Pursuit-Evasion Hybrid Spacecraft Controllers
AIAA | 2015
|Numerical Solution of the Three-Dimensional Orbital Pursuit — Evasion Game
Online Contents | 2009
|