This work explores reinforcement learning (RL) for on-board planning and scheduling of an agile Earth-observing satellite (AEOS). In this formulation of the AEOS scheduling problem, a spacecraft in low Earth orbit attempts to maximize the weighted sum of targets collected and downlinked. The AEOS scheduling problem is formulated as a Markov decision process (MDP) where the number of upcoming imaging targets included in the action space is an adjustable parameter to account for clusters of imaging targets with varying priorities. Monte Carlo tree search (MCTS) and supervised learning are used to train a set of agents with varying numbers of targets in the action space. Two backup strategies are explored for MCTS—an incremental averaging operator and a maximization operator. For all backup operators, performance asymptotically increases as the number of targets in the action space approaches the maximum number of available targets. A benchmark is computed with MCTS to determine an upper bound on performance. Furthermore, MCTS is compared with solutions generated by a genetic algorithm. MCTS demonstrates a 2%–5% increase in average reward at 10%–20% of the single-core wall clock time of the genetic algorithm. A search of various neural network hyperparameters is presented, and the trained neural networks are shown to approximate the MCTS policy with three orders of magnitude less execution time. Finally, the trained agents and the genetic algorithm are deployed on varying target densities for comparison purposes and to demonstrate robustness to mission profiles outside of the training distribution.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Reinforcement Learning for the Agile Earth-Observing Satellite Scheduling Problem


    Contributors:


    Publication date :

    2023-10-01


    Size :

    1856377 byte




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English



    Scheduling an Agile Multipayload Earth-Observing Satellite

    Zhang, Wenyuan / Zheng, Gangtie | AIAA | 2024


    REINFORCEMENT LEARNING FOR THE MULTI-SATELLITE EARTH-OBSERVING SCHEDULING PROBLEM

    Herrmann, Adam / Carneiro, Jo˜ao Vaz / Schaub, Hanspeter | Springer Verlag | 2024


    Optimization-Based Scheduling Method for Agile Earth-Observing Satellite Constellation

    Cho, Doo-Hyun / Kim, Jun-Hong / Choi, Han-Lim et al. | AIAA | 2018

    Free access