The invention relates to a spacecraft cluster game hunting motion planning method based on multi-agent reinforcement learning, and the method comprises the steps: obtaining the communication constraint in a cluster according to the structure of a spacecraft cluster system and the communication performance limitation of a cluster sub-machine; obtaining an action space and a state space of the spacecraft cluster according to a hunting distance requirement and communication constraints in the cluster; according to a completion target and a termination condition of the task of the spacecraft cluster system, obtaining a composite reward function; according to a joint action-value function and the composite reward function in a multi-agent reinforcement learning algorithm, obtaining a segmented training strategy for an exploration stage, a strategy optimization stage and a strategy stabilization stage; and obtaining a spacecraft cluster game hunting motion planning strategy network according to the motion space and the state space of the spacecraft cluster, the composite reward function and the segmented training strategy. According to the spacecraft cluster game hunting motion planning method provided by the embodiment of the invention, effective game hunting of non-cooperative spacecrafts by a spacecraft cluster can be realized, and while the hunting task success rate of a cluster system is ensured, the strategy output network expansibility can be used for realizing the game hunting of the non-cooperative spacecrafts. The cluster game hunting planning strategy output network has good universality, and reference can be provided for space cluster confrontation decision-making in a dynamic environment.
本发明涉及一种基于多智能体强化学习的航天器集群博弈围捕运动规划方法,包括:依据航天器集群系统结构和集群子机通信性能限制,获得集群内部的通信约束;依据围捕距离要求与所述集群内部的通信约束,获得航天器集群的动作空间、状态空间;依据航天器集群系统任务的完成目标及终止条件,获得复合型奖励函数;依据多智能体强化学习算法中的联合动作‑价值函数和所述复合型奖励函数,获得针对探索阶段、策略优化阶段和策略稳定阶段的分段训练策略;依据所述航天器集群的动作空间及状态空间、复合型奖励函数和分段训练策略,获得航天器集群博弈围捕运动规划策略网络。根据本发明实例中提供的航天器集群博弈围捕运动规划方法,可实现航天器集群对非合作航天器的有效博弈围捕,在保证集群系统围捕任务成功率的同时,通过策略输出网络的可拓展性,使集群博弈围捕规划策略输出网络拥有良好的泛用性,可为动态环境下的空间集群对抗决策提供参考。
Spacecraft cluster game hunting motion planning method based on multi-agent reinforcement learning
一种基于多智能体强化学习的航天器集群博弈围捕运动规划方法
2024-05-21
Patent
Electronic Resource
Chinese
A Collaborative Control Method for Spacecraft Clusters Based on Multi Agent Reinforcement Learning
Springer Verlag | 2025
|British Library Conference Proceedings | 2005
|MODEL BASED MULTI-AGENT NUMERIC PLANNING SYSTEM FOR SPACECRAFT
TIBKAT | 2021
|