In this chapter, we show how to use reinforcement learning (RL) to generate a control policy for the same uncertain dynamical system considered in the previous chapters. Specifically, we present an online model-based RL (MBRL) algorithm that balances the often competing objectives of learning and safety by simultaneously learning the value function of an optimal control problem and the uncertain parameters of a dynamical system using real-time data. Central to our approach is a safe exploration framework based on the adaptive control barrier functions introduced in Chap. 5. We start by formulating an infinite-horizon optimal control problem as an RL problem in Sect. 8.1. Approximations for the value function used in the RL algorithm are discussed in Sect. 8.2. The main part of this chapter is Sect. 8.3, where we present the online MBRL algorithm. We illustrate the method with numerical examples in Sect. 8.4 and conclude with final remarks, references, and suggestions for further reading in Sect. 8.5.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Safe Exploration in Model-Based Reinforcement Learning


    Additional title:

    synth. Lectures on Computer sci.


    Contributors:
    Cohen, Max (author) / Belta, Calin (author)


    Publication date :

    2023-05-16


    Size :

    31 pages





    Type of media :

    Article/Chapter (Book)


    Type of material :

    Electronic Resource


    Language :

    English




    SHERPA: a safe exploration algorithm for Reinforcement Learning controllers

    Mannucci, Tommaso / Van Kampen, Erik-Jan / de Visser, Coen C. et al. | AIAA | 2015


    Human Demonstrations for Fast and Safe Exploration in Reinforcement Learning

    Schonebaum, Gerben / Junell, Jaime / Van Kampen, Erik-Jan | AIAA | 2017


    Safe robot execution in model-based reinforcement learning

    Martínez Martínez, David / Alenyà Ribas, Guillem / Torras, Carme | BASE | 2015

    Free access

    Safe robot execution in model-based reinforcement learning

    Martínez, David / Alenyà, Guillem / Torras, Carme | BASE | 2015

    Free access

    Safe Trajectory Sampling in Model-Based Reinforcement Learning

    Zwane, S / Hadjivelichkov, D / Luo, Y et al. | BASE | 2023

    Free access