Advances in request scheduling algorithms are necessary to fully utilize the capabilities of agile Earth-observing satellites. While typical current approaches preplan sequences of image collections for a static set of requests, onboard autonomous methods that select requests to fulfill on-the-fly can adapt to opportunistic events and reduce the burden on operators. Prior approaches that utilized reinforcement learning for autonomy considered fixed-duration decision intervals, which limited performance over high-density requests and oversimplified mission operations. In this paper, reinforcement learning is used to derive policies that make decisions at variable intervals to improve performance by scheduling requests in a mission-like manner. The resulting policies perform competitively against optimal global schedules in a flight-like evaluation environment. This is achieved by training on a new semi-Markov decision process (sMDP) formulation of the problem, leveraging step-duration information when learning. Ablation studies over new sMDP-specific training algorithm modifications and observation spaces are performed to learn a highly performant policy. Benefits of the method, including the implicit ability to account for resource constraints, are demonstrated. Ultimately, variable-decision-interval reinforcement-learning-based policies are established as a viable method for autonomous Earth-observing satellite scheduling under mission-realistic conditions, providing a closed-loop and computationally inexpensive alternative to traditional preplanning methods.
Learning Policies for Autonomous Earth-Observing Satellite Scheduling over Semi-Markov Decision Processes
01.05.2025
Aufsatz (Konferenz) , Aufsatz (Zeitschrift)
Elektronische Ressource
Englisch
REINFORCEMENT LEARNING FOR THE MULTI-SATELLITE EARTH-OBSERVING SCHEDULING PROBLEM
Springer Verlag | 2024
|An autonomous earth-observing sensorWeb
Tema Archiv | 2005
|Optimization-Based Scheduling Method for Agile Earth-Observing Satellite Constellation
AIAA | 2018
|