Model-based reinforcement learning aims to learn a policy to solve a target task by leveraging a learned dynamics model. This approach, paired with principled handling of uncertainty allows for data-efficient policy learning in robotics. However, the physical environment has feasibility and safety constraints that need to be incorporated into the policy before it is safe to execute on a real robot. In this work, we study how to enforce the aforementioned constraints in the context of model-based reinforcement learning with probabilistic dynamics models. In particular, we investigate how trajectories sampled from the learned dynamics model can be used on a real robot, while fulfilling user-specified safety requirements. We present a model-based reinforcement learning approach using Gaussian processes where safety constraints are taken into account without simplifying Gaussian assumptions on the predictive state distributions. We evaluate the proposed approach on different continuous control tasks with varying complexity and demonstrate how our safe trajectory-sampling approach can be directly used on a real robot without violating safety constraints.


    Access

    Download


    Export, share and cite



    Title :

    Safe Trajectory Sampling in Model-Based Reinforcement Learning


    Contributors:
    Zwane, S (author) / Hadjivelichkov, D (author) / Luo, Y (author) / Bekiroglu, Y (author) / Kanoulas, D (author) / Deisenroth, MP (author)

    Publication date :

    2023-01-01


    Remarks:

    In: IEEE International Conference on Automation Science and Engineering. IEEE: Auckland, New Zealand. (2023)


    Type of media :

    Paper


    Type of material :

    Electronic Resource


    Language :

    English



    Classification :

    DDC:    629



    SRL-TR2: A Safe Reinforcement Learning Based TRajectory TRacker Framework

    Wang, Chengyu / Wang, Luhan / Lu, Zhaoming et al. | IEEE | 2023



    Safe Exploration in Model-Based Reinforcement Learning

    Cohen, Max / Belta, Calin | Springer Verlag | 2023


    Safe Reinforcement Learning for Single Train Trajectory Optimization via Shield SARSA

    Zhao, Zicong / Xun, Jing / Wen, Xuguang et al. | IEEE | 2023


    Safe robot execution in model-based reinforcement learning

    Martínez Martínez, David / Alenyà Ribas, Guillem / Torras, Carme | BASE | 2015

    Free access