Advances in request scheduling algorithms are necessary to fully utilize the capabilities of agile Earth-observing satellites. While typical current approaches preplan sequences of image collections for a static set of requests, onboard autonomous methods that select requests to fulfill on-the-fly can adapt to opportunistic events and reduce the burden on operators. Prior approaches that utilized reinforcement learning for autonomy considered fixed-duration decision intervals, which limited performance over high-density requests and oversimplified mission operations. In this paper, reinforcement learning is used to derive policies that make decisions at variable intervals to improve performance by scheduling requests in a mission-like manner. The resulting policies perform competitively against optimal global schedules in a flight-like evaluation environment. This is achieved by training on a new semi-Markov decision process (sMDP) formulation of the problem, leveraging step-duration information when learning. Ablation studies over new sMDP-specific training algorithm modifications and observation spaces are performed to learn a highly performant policy. Benefits of the method, including the implicit ability to account for resource constraints, are demonstrated. Ultimately, variable-decision-interval reinforcement-learning-based policies are established as a viable method for autonomous Earth-observing satellite scheduling under mission-realistic conditions, providing a closed-loop and computationally inexpensive alternative to traditional preplanning methods.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Learning Policies for Autonomous Earth-Observing Satellite Scheduling over Semi-Markov Decision Processes




    Publication date :

    2025-05-01




    Type of media :

    Conference paper , Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English



    REINFORCEMENT LEARNING FOR THE MULTI-SATELLITE EARTH-OBSERVING SCHEDULING PROBLEM

    Herrmann, Adam / Carneiro, Jo˜ao Vaz / Schaub, Hanspeter | Springer Verlag | 2024


    Scheduling an Agile Multipayload Earth-Observing Satellite

    Zhang, Wenyuan / Zheng, Gangtie | AIAA | 2024



    An autonomous earth-observing sensorWeb

    Chien, S. / Cichy, B. / Davies, A. et al. | Tema Archive | 2005


    Optimization-Based Scheduling Method for Agile Earth-Observing Satellite Constellation

    Cho, Doo-Hyun / Kim, Jun-Hong / Choi, Han-Lim et al. | AIAA | 2018

    Free access