Deep reinforcement learning (DRL) is a promising method to learn control policies for robots only from demonstration and experience. To cover the whole dynamic behaviour of the robot, DRL training is an active exploration process typically performed in simulation environments. Although this simulation training is cheap and fast, applying DRL algorithms to real-world settings is difficult. If agents are trained until they perform safely in simulation, transferring them to physical systems is difficult due to the sim-to-real gap caused by the difference between the simulation dynamics and the physical robot. In this paper, we present a method of online training a DRL agent to drive autonomously on a physical vehicle by using a model-based safety supervisor. Our solution uses a supervisory system to check if the action selected by the agent is safe or unsafe and ensure that a safe action is always implemented on the vehicle. With this, we can bypass the sim-to-real problem while training the DRL algorithm safely, quickly, and efficiently. We compare our method with conventional learning in simulation and on a physical vehicle. We provide a variety of real-world experiments where we train online a small-scale vehicle to drive autonomously with no prior simulation training. The evaluation results show that our method trains agents with improved sample efficiency while never crashing, and the trained agents demonstrate better driving performance than those trained in simulation.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    Bypassing the Simulation-to-Reality Gap: Online Reinforcement Learning Using a Supervisor


    Beteiligte:


    Erscheinungsdatum :

    2023-12-05


    Format / Umfang :

    804074 byte





    Medientyp :

    Aufsatz (Konferenz)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch



    Bypassing bottlenecks

    Online Contents | 1994


    A real time digital simulation supervisor

    Cleveland, J. I., II | NTRS | 1970


    Weigh Station Bypassing

    H. F. Southgate / J. D. Crabtree / J. G. Pigman et al. | NTIS | 1993


    Bypassing weigh stations

    Online Contents | 1997


    Automotive Entwicklung durch Bypassing

    Lieberwirth, Ralf | Tema Archiv | 2005