Abstract To overcome the time-consuming training caused by the sparse reward function in reinforcement learning, an efficient dense reward framework for robust low-thrust trajectory optimization is proposed. The dense reward functions are designed separately for the deterministic and considered uncertain scenarios, including state uncertainties, observation uncertainties and execution uncertainties. For the uncertainties, the dense reward function is designed to diminish the deviation with respect to the isochronous nominal state along the corresponding deterministic optimal trajectory at each step, rendering the reward function no longer sparse and suitable for complex multirevolution problems. In addition, a multistage reward function of the terminal constraints for the rendezvous missions is designed by incorporating some exponential acceleration terms, enabling significant improvement in training efficiency as the terminal errors become low. In addition, a dense reward function for the deterministic scenario is also proposed via the introduction of empirical forbidden zones and an exponential term. The effectiveness and efficiency of the proposed method is demonstrated in a simple Earth-Mars mission and a complex Earth-Venus multirevolution mission. The promising results verify the significant effect of the proposed method in speeding up the process of training an initial incapable agent to an ‘expert’ while guaranteeing or even improving the performance.
Densely rewarded reinforcement learning for robust low-thrust trajectory optimization
Advances in Space Research ; 72 , 4 ; 964-981
2023-03-31
18 pages
Article (Journal)
Electronic Resource
English
Approximate finite-thrust trajectory optimization.
AIAA | 1969
|Emerald Group Publishing | 2000
Low-Thrust, High-Accuracy Trajectory Optimization
Online Contents | 2007
|Low-Thrust, High-Accuracy Trajectory Optimization
AIAA | 2007
|