Abstract Closed-loop feedback-driven control laws can be used to solve low-thrust many-revolution trajectory design and guidance problems with minimal computational cost. Lyapunov-based control laws offer the benefits of increased stability whilst their optimality can be increased by tuning their parameters. In this paper, a reinforcement learning framework is used to make the parameters of the Lyapunov-based Q-law state-dependent, increasing its optimality. The Jacobian of these state-dependent parameters is available analytically and, unlike in other optimisation approaches, can be used to enforce stability throughout the transfer. The results focus on GTO–GEO and LEO–GEO transfers in Keplerian dynamics, including the effects of eclipses. The impact of the network architecture on the behaviour is investigated for both time- and mass-optimal transfers. Robustness to navigation errors and thruster misalignment is demonstrated using Monte Carlo analyses. The resulting approach offers potential for on-board autonomous transfers and orbit reconfiguration.

    Highlights Optimality of Lyapunov-based Q-laws is improved using state-dependent parameters. Highlight the robust combination of Reinforcement Learning and Lyapunov control. Lyapunov stability is ensured analytically through the state-weight Jacobian. Low-thrust time and mass-optimal GTO–GEO and LEO–GEO transfers are investigated. Close-to-optimal controller remains closed-loop with potential for onboard use. Monte Carlo simulations with navigation errors and thruster misalignment.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Optimal Q-laws via reinforcement learning with guaranteed stability


    Contributors:

    Published in:

    Acta Astronautica ; 187 ; 511-528


    Publication date :

    2021-07-07


    Size :

    18 pages




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English




    Guaranteed Globally Optimal continuous Reinforcement Learning (AIAA 2014-0010)

    Bijl, H. / Van Kampen, E.-J. / Chu, Q.P. et al. | British Library Conference Proceedings | 2014



    Optimal Adaptive Cruise Control with Guaranteed String Stability

    Liang, C.-Y. / Peng, H. | British Library Conference Proceedings | 1999


    Optimal Adaptive Cruise Control with Guaranteed String Stability

    Liang, C. / Peng, H. / Society of Automotive Engineers of Japan | British Library Conference Proceedings | 1998


    Optimal adaptive cruise control with guaranteed string stability

    Liang, C.Y. / Peng, H. | Tema Archive | 1998