The invention relates to a spacecraft cluster game hunting motion planning method based on multi-agent reinforcement learning, and the method comprises the steps: obtaining the communication constraint in a cluster according to the structure of a spacecraft cluster system and the communication performance limitation of a cluster sub-machine; obtaining an action space and a state space of the spacecraft cluster according to a hunting distance requirement and communication constraints in the cluster; according to a completion target and a termination condition of the task of the spacecraft cluster system, obtaining a composite reward function; according to a joint action-value function and the composite reward function in a multi-agent reinforcement learning algorithm, obtaining a segmented training strategy for an exploration stage, a strategy optimization stage and a strategy stabilization stage; and obtaining a spacecraft cluster game hunting motion planning strategy network according to the motion space and the state space of the spacecraft cluster, the composite reward function and the segmented training strategy. According to the spacecraft cluster game hunting motion planning method provided by the embodiment of the invention, effective game hunting of non-cooperative spacecrafts by a spacecraft cluster can be realized, and while the hunting task success rate of a cluster system is ensured, the strategy output network expansibility can be used for realizing the game hunting of the non-cooperative spacecrafts. The cluster game hunting planning strategy output network has good universality, and reference can be provided for space cluster confrontation decision-making in a dynamic environment.

    本发明涉及一种基于多智能体强化学习的航天器集群博弈围捕运动规划方法,包括:依据航天器集群系统结构和集群子机通信性能限制,获得集群内部的通信约束;依据围捕距离要求与所述集群内部的通信约束,获得航天器集群的动作空间、状态空间;依据航天器集群系统任务的完成目标及终止条件,获得复合型奖励函数;依据多智能体强化学习算法中的联合动作‑价值函数和所述复合型奖励函数,获得针对探索阶段、策略优化阶段和策略稳定阶段的分段训练策略;依据所述航天器集群的动作空间及状态空间、复合型奖励函数和分段训练策略,获得航天器集群博弈围捕运动规划策略网络。根据本发明实例中提供的航天器集群博弈围捕运动规划方法,可实现航天器集群对非合作航天器的有效博弈围捕,在保证集群系统围捕任务成功率的同时,通过策略输出网络的可拓展性,使集群博弈围捕规划策略输出网络拥有良好的泛用性,可为动态环境下的空间集群对抗决策提供参考。


    Access

    Download


    Export, share and cite



    Title :

    Spacecraft cluster game hunting motion planning method based on multi-agent reinforcement learning


    Additional title:

    一种基于多智能体强化学习的航天器集群博弈围捕运动规划方法


    Contributors:
    CHEN GANG (author) / CUI ZI'ANG (author) / LIU YUQIANG (author) / YOU HONG (author) / LI TONG (author) / ZHOU YIDI (author)

    Publication date :

    2024-05-21


    Type of media :

    Patent


    Type of material :

    Electronic Resource


    Language :

    Chinese


    Classification :

    IPC:    B64G Raumfahrt , COSMONAUTICS / G06F ELECTRIC DIGITAL DATA PROCESSING , Elektrische digitale Datenverarbeitung / G06N COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS , Rechnersysteme, basierend auf spezifischen Rechenmodellen



    A Collaborative Control Method for Spacecraft Clusters Based on Multi Agent Reinforcement Learning

    Liang, Xi / Wei, Cheng / Zhao, Jianbo et al. | Springer Verlag | 2025


    Cloud game computing offload based on Multi-Agent Reinforcement Learning

    Tian, Kaicong / Liu, Yitong / Yang, Hongwen et al. | IEEE | 2022


    Adaptive Task Flow Management of Distributed Spacecraft Bus based on Multi Agent Reinforcement Learning

    Eishima, T. / Nakasuka, S. / European Space Agency | British Library Conference Proceedings | 2005



    MODEL BASED MULTI-AGENT NUMERIC PLANNING SYSTEM FOR SPACECRAFT

    Zhao, Yuting / Xu, Rui / Li, Zhaoyu et al. | TIBKAT | 2021