This paper proposes two novel adaptive optimal control algorithms for continuous-time nonlinear affine systems based on reinforcement learning: i) generalised policy iteration (GPI) and ii) Q-learning. As a result, the a priori knowledge of the system drift f(x) is not needed via GPI, which gives us a partially model-free and online solution. We then for the first time extend the idea of Q-learning to the nonlinear continuoustime optimal control problem in a noniterative manner. This leads to a completely model-free method where neither the system drift f(x) nor the input gain g(x) is needed. For both methods, the adaptive critic and actor are continuously and simultaneously updating each other without iterative steps, which effectively avoids the hybrid structure and the need for an initial stabilising control policy. Moreover, finite-time convergence is guaranteed by using a sliding mode technique in the new adaptive approach, where the persistent excitation (PE) condition can be directly verified online. We also prove the overall Lyapunov stability and demonstrate the effectiveness of the proposed algorithms using numerical examples.


    Zugriff

    Download


    Exportieren, teilen und zitieren



    Titel :

    Adaptive Optimal Control via Continuous-Time Q-Learning for Unknown Nonlinear Affine Systems


    Beteiligte:

    Erscheinungsdatum :

    2020-03-12


    Anmerkungen:

    Chen , A S & Herrmann , G 2020 , Adaptive Optimal Control via Continuous-Time Q-Learning for Unknown Nonlinear Affine Systems . in 58th Conference on Decision and Control . 58th IEEE Conference on Decision and Control , Nice , France , 11/12/19 . https://doi.org/10.1109/CDC40024.2019.9030116



    Medientyp :

    Aufsatz (Zeitschrift)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch


    Klassifikation :

    DDC:    629



    Optimal Plug-and-Control of Unknown Nonlinear Systems

    Adelberger, Daniel / Giarré, Laura / Ohtsuka, Toshiyuki et al. | BASE | 2022

    Freier Zugriff

    Adaptive output feedback neural network control of uncertain non-affine systems with unknown control direction

    Arefi, Mohammad M. / Zarei, Jafar / KARIMI, HAMID REZA | BASE | 2014

    Freier Zugriff

    Event-triggered optimal control of completely unknown nonlinear systems via identifier-critic learning

    Peng, Zhinan / Zhang, Zhiquan / Luo, Rui et al. | BASE

    Freier Zugriff

    Adaptive Sliding Mode Control for Non-Affine Nonlinear Vehicle Systems

    Ahn, Chaeik / Kim, Hyounjin / Kim, Youdan | AIAA | 2007