Driving an Autonomous Vehicle (AV) in dynamic traffic is a critical task, as the overtaking maneuver being considered one of the most complex due to involvement of several sub-maneuvers. Recent advances in Deep Reinforcement Learning (DRL) have resulted in AVs exhibiting exceptional performance in addressing overtaking-related challenges. However, the intricate nature of the overtaking presents difficulties for a RL agent to proficiently handle all its sub-maneuvers that include left lane change, right lane change and straight drive. Furthermore, the dynamic traffic restricts the RL agents to execute the sub-maneuvers at critical checkpoints involved in overtaking. To address this, we propose an approach inspired by semi-Markov options, called Dynamic Option Policy enabled Hierarchical Deep Reinforcement Learning (DOP-HDRL). This innovative approach allows the selection of sub-maneuver agents using a single dynamic option policy, while employing individual DRL agents specifically trained for each sub-maneuver to perform tasks during overtaking in dynamic environments. By breaking down overtaking maneuvers into several sub-maneuvers and controlling them using a single policy, the DOP-HDRL approach reduces training time and computational load compared to classical DRL agents. Moreover, DOP-HDRL easily integrates basic traffic safety rules into overtaking maneuvers to offer more robust solutions. The DOP-HDRL approach is rigorously evaluated through multiple overtaking and non-overtaking scenarios inspired by the National Highway Traffic Safety Administration (NHTSA) pre-crash scenarios in the CARLA simulator. On an average, the DOP-HDRL approach shows 100% completion rate, 14% least collision rate, 25% optimal clearance distance, and 7% more average speed compared to the state-of-the-art methods.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Dynamic Option Policy Enabled Hierarchical Deep Reinforcement Learning Model for Autonomous Overtaking Maneuver


    Contributors:


    Publication date :

    2025-04-01


    Size :

    2148170 byte




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English



    An Autonomous Overtaking Maneuver Based on Relative Position Information

    Zhang, Meihong / Zhang, Tingting / Zhang, Qinyu | IEEE | 2018


    Overtaking maneuver scenario building for autonomous vehicles with PreScan software

    Josue Ortega / Henrietta Lengyel / Zsolt Szalay | DOAJ | 2020

    Free access

    Deep Reinforcement Learning for Autonomous Aerobraking Maneuver Planning

    Falcone, Giusy / Putnam, Zachary R. | TIBKAT | 2022


    Deep Reinforcement Learning for Autonomous Aerobraking Maneuver Planning

    Falcone, Giusy / Putnam, Zachary R. | AIAA | 2022