Abstract To overcome the time-consuming training caused by the sparse reward function in reinforcement learning, an efficient dense reward framework for robust low-thrust trajectory optimization is proposed. The dense reward functions are designed separately for the deterministic and considered uncertain scenarios, including state uncertainties, observation uncertainties and execution uncertainties. For the uncertainties, the dense reward function is designed to diminish the deviation with respect to the isochronous nominal state along the corresponding deterministic optimal trajectory at each step, rendering the reward function no longer sparse and suitable for complex multirevolution problems. In addition, a multistage reward function of the terminal constraints for the rendezvous missions is designed by incorporating some exponential acceleration terms, enabling significant improvement in training efficiency as the terminal errors become low. In addition, a dense reward function for the deterministic scenario is also proposed via the introduction of empirical forbidden zones and an exponential term. The effectiveness and efficiency of the proposed method is demonstrated in a simple Earth-Mars mission and a complex Earth-Venus multirevolution mission. The promising results verify the significant effect of the proposed method in speeding up the process of training an initial incapable agent to an ‘expert’ while guaranteeing or even improving the performance.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Densely rewarded reinforcement learning for robust low-thrust trajectory optimization


    Contributors:
    Hu, Jincheng (author) / Yang, Hongwei (author) / Li, Shuang (author) / Zhao, Yingjie (author)

    Published in:

    Advances in Space Research ; 72 , 4 ; 964-981


    Publication date :

    2023-03-31


    Size :

    18 pages




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English





    Success rewarded

    Emerald Group Publishing | 2000


    Low-Thrust, High-Accuracy Trajectory Optimization

    Ross, I.M. | Online Contents | 2007


    Low-Thrust, High-Accuracy Trajectory Optimization

    I. Michael Ross / Qi Gong / Pooya Sekhavat | AIAA | 2007


    Robust Low-Thrust Trajectory Optimization Using Convex Programming and a Homotopic Approach

    Morelli, Andrea Carlo / Hofmann, Christian / Topputo, Francesco | IEEE | 2022