Planning safe trajectories under uncertain and dynamic conditions makes the autonomous driving problem significantly complex. Current heuristic-based algorithms such as the slot-based method rely heavily on hand-engineered parameters and are restricted to specific scenarios. Supervised learning methods such as Imitation Learning lack generalization and safety guarantees. To address these problems and to ensure a robust framework, we propose a Robust-Hierarchical Reinforcement Learning (HRL) framework for learning autonomous driving policies. We adapt a state-of-the-art algorithm, Hierarchical Double Deep Q-learning (h-DDQN), and make the framework robust by (1) constituting the decision of selecting driving maneuver as a high-level option; (2) for the lower-level controller, outputting waypoint trajectories to track with a Proportional-Integral-Derivative (PID) controller instead of direct acceleration/steering actions; and (3) using a Long-Short-Term-Memory (LSTM) layer in the network to alleviate the effects of observation noise and dynamic driving behaviors. Moreover, to improve the sample efficiency, we use Hybrid Reward Mechanism and Reward-Driven Exploration. Results from the high-fidelity CARLA simulator while simulating different interactive lane change scenarios indicate that the proposed framework reduces convergence time, generates smoother trajectories, and can better handle dynamic surroundings and noisy observations as compared to other traditional RL approaches.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Trajectory Planning for Autonomous Vehicles Using Hierarchical Reinforcement Learning


    Contributors:


    Publication date :

    2021-09-19


    Size :

    714592 byte




    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English




    A Trajectory Simulation Approach for Autonomous Vehicles Path Planning using Deep Reinforcement Learning

    de Oliveira Lima, Jean Phelipe / Oliveira, Raimundo Correa de / Costa, Cleinaldo de Almeida | BASE | 2020

    Free access

    OFF-ROAD AUTONOMOUS VEHICLES TRAVERSABILITY ANALYSIS AND TRAJECTORY PLANNING BASED ON DEEP INVERSE REINFORCEMENT LEARNING

    Zhu, Zeyu / Li, Nan / Sun, Ruoyu et al. | British Library Conference Proceedings | 2020



    Vertical Trajectory Planning for Autonomous Vehicles

    Jurisch, Matthias / Koch, Thorsten | TIBKAT | 2021