In this chapter, we show how to use reinforcement learning (RL) to generate a control policy for the same uncertain dynamical system considered in the previous chapters. Specifically, we present an online model-based RL (MBRL) algorithm that balances the often competing objectives of learning and safety by simultaneously learning the value function of an optimal control problem and the uncertain parameters of a dynamical system using real-time data. Central to our approach is a safe exploration framework based on the adaptive control barrier functions introduced in Chap. 5. We start by formulating an infinite-horizon optimal control problem as an RL problem in Sect. 8.1. Approximations for the value function used in the RL algorithm are discussed in Sect. 8.2. The main part of this chapter is Sect. 8.3, where we present the online MBRL algorithm. We illustrate the method with numerical examples in Sect. 8.4 and conclude with final remarks, references, and suggestions for further reading in Sect. 8.5.
Safe Exploration in Model-Based Reinforcement Learning
synth. Lectures on Computer sci.
Adaptive and Learning-Based Control of Safety-Critical Systems ; Kapitel : 8 ; 133-163
2023-05-16
31 pages
Aufsatz/Kapitel (Buch)
Elektronische Ressource
Englisch
Temporal Logic Guided Safe Model-Based Reinforcement Learning
Springer Verlag | 2023
|