This paper aims to solve the trajectory tracking control problem for an autonomous vehicle based on reinforcement learning methods. Existing reinforcement learning approaches have found limited successful applications on safety-critical tasks in the real world mainly due to two challenges: 1) sim-to-real transfer; 2) closed-loop stability and safety concern. In this paper, we propose an actor-critic-style framework SRL-TR2, in which the RL-based TRajectory TRackers are trained under the safety constraints and then deployed to a full-size vehicle as the lateral controller. To improve the generalization ability, we adopt a light-weight adapter State and Action Space Alignment (SASA) to establish mapping relations between the simulation and reality. To address the safety concern, we leverage an expert strategy to take over the control when the safety constraints are not satisfied. Hence, we conduct safe explorations during the training process and improve the stability of the policy. The experiments show that our agents can achieve one-shot transfer across simulation scenarios and unseen realistic scenarios, finishing the field tests with average running time less than 10 ms/step and average lateral error less than 0.1 m under the speed ranging from 12 km/h to 18 km/h. A video of the field tests is available at https://youtu.be/pjWcN_fV24g.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    SRL-TR2: A Safe Reinforcement Learning Based TRajectory TRacker Framework


    Contributors:
    Wang, Chengyu (author) / Wang, Luhan (author) / Lu, Zhaoming (author) / Chu, Xinghe (author) / Shi, Zhengrui (author) / Deng, Jiayin (author) / Su, Tianyang (author) / Shou, Guochu (author) / Wen, Xiangming (author)


    Publication date :

    2023-06-01


    Size :

    4094164 byte




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English



    Deep Reinforcement Learning based Aggressive Flight Trajectory Tracker

    Shadeed, Omar / Hasanzade, Mehmet / Koyuncu, Emre | AIAA | 2021


    DEEP REINFORCEMENT LEARNING BASED AGGRESSIVE FLIGHT TRAJECTORY TRACKER

    Shadeed, Omar / Hasanzade, Mehmet / Koyuncu, Emre | TIBKAT | 2021


    Safe Trajectory Sampling in Model-Based Reinforcement Learning

    Zwane, S / Hadjivelichkov, D / Luo, Y et al. | BASE | 2023

    Free access

    TRACKER TRAJECTORY VALIDATION

    CLAWSON TAYLOR SCOTT / FILARSKY BRIAN MICHAEL | European Patent Office | 2024

    Free access

    TRACKER TRAJECTORY VALIDATION

    CLAWSON TAYLOR SCOTT / FILARSKY BRIAN MICHAEL | European Patent Office | 2024

    Free access