One of the major challenges in Deep Reinforcement Learning for control is the need for extensive training to learn the policy. Motivated by this, we present the design of the Control-Tutored Deep Q-Networks (CT-DQN) algorithm, a Deep Reinforcement Learning algorithm that leverages a control tutor, i.e., an exogenous control law, to reduce learning time. The tutor can be designed using an approximate model of the system, without any assumption about the knowledge of the system’s dynamics. There is no expectation that it will be able to achieve the control objective if used stand-alone. During learning, the tutor occasionally suggests an action, thus partially guiding exploration. We validate our approach on three scenarios from OpenAI Gym: the inverted pendulum, lunar lander, and car racing. We demonstrate that CT-DQN is able to achieve better or equivalent data efficiency with respect to the classic function approximation solutions.


    Access

    Download


    Export, share and cite



    Title :

    CT-DQN: Control-Tutored Deep Reinforcement Learning


    Contributors:

    Publication date :

    2023-06-16


    Remarks:

    In: Proceedings of The 5th Annual Learning for Dynamics and Control Conference, Volume 211. (pp. pp. 941-953). PMLR: Philadelphia, PA, USA. (2023)


    Type of media :

    Paper


    Type of material :

    Electronic Resource


    Language :

    English



    Classification :

    DDC:    629



    Powerslide Control with Deep Reinforcement Learning

    Jaumann, Florian / Schuster, Tobias / Unterreiner, Michael et al. | Springer Verlag | 2024

    Free access

    Universal Quantum Control through Deep Reinforcement Learning

    Niu, Murphy Yuezhen / Boixo, Sergio / Smelyanskiy, Vadim N. et al. | AIAA | 2019


    DEEP REINFORCEMENT LEARNING FOR EVTOL HOVERING CONTROL

    Alarcon, D. S. / Bidinotto, J. H. | British Library Conference Proceedings | 2022


    Universal Quantum Control through Deep Reinforcement Learning

    Niu, Murphy Yuezhen / Boixo, Sergio / Smelyanskiy, Vadim N. et al. | TIBKAT | 2019


    Adaptive Traffic Control Using Deep Reinforcement Learning

    N, Dhashyanth / R, Hemchand / R, Priyanga et al. | IEEE | 2024