As connectivity and sensing technologies become more mature, automated vehicles can predict future driving situations and utilize this information to drive more energy-efficiently than human-driven vehicles. However, future information beyond the limited connectivity and sensing range is difficult to predict and utilize, limiting the energy-saving potential of energy-efficient driving. Thus, we combine a conventional speed optimization planner, developed in our previous work, and reinforcement learning to propose a real-time intelligent speed optimization planner for connected and automated vehicles. We briefly summarize the conventional speed optimization planner with limited information, based on closed-form energy-optimal solutions, and present its multiple parameters that determine reference speed trajectories. Then, we use a deep reinforcement learning (DRL) algorithm, such as a deep Q-learning algorithm, to find the policy of how to adjust these parameters in real-time to dynamically changing situations in order to realize the full potential of energy-efficient driving. The model-free DRL algorithm, based on the experience of the system, can learn the optimal policy through iteratively interacting with different driving scenarios without increasing the limited connectivity and sensing range. The training process of the parameter adaptation policy exploits a high-fidelity simulation framework that can simulate multiple vehicles with full powertrain models and the interactions between vehicles and their environment. We consider intersection-approaching scenarios where there is one traffic light with different signal phase and timing setup. Results show that the learned optimal policy enables the proposed intelligent speed optimization planner to properly adjust the parameters in a piecewise constant manner, leading to additional energy savings without increasing total travel time compared to the conventional speed optimization planner.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    A Real-Time Intelligent Speed Optimization Planner Using Reinforcement Learning


    Additional title:

    Sae Technical Papers


    Contributors:

    Conference:

    SAE WCX Digital Summit ; 2021



    Publication date :

    2021-04-06




    Type of media :

    Conference paper


    Type of material :

    Print


    Language :

    English




    Highway Exiting Planner for Automated Vehicles Using Reinforcement Learning

    Cao, Zhong / Yang, Diange / Xu, Shaobing et al. | IEEE | 2021


    Real-Time Intelligent Autonomous Intersection Management Using Reinforcement Learning

    Gunarathna, Udesh / Karunasekera, Shanika / Borovica-Gajic, Renata et al. | IEEE | 2022


    M2M-enabled real-time Trip Planner

    Cerritos, Eduardo / Lin, Fuchun Joseph | IEEE | 2014


    Intelligent Engine Activation Planner

    WRAY KYLE HOLLINS / PEDERSEN LIAM / LUI RICHARD et al. | European Patent Office | 2023

    Free access

    Intelligent Engine Activation Planner

    WRAY KYLE HOLLINS / PEDERSEN LIAM / LUI RICHARD et al. | European Patent Office | 2022

    Free access