The large-scale popularization of electric buses (EBs) marks a significant stride in sustainable development strategies aimed at environmental conservation. A critical concern for bus companies is reducing the operational costs associated with charging these vehicles. This task is particularly challenging due to the uncertainties in travel time, energy consumption, and fluctuating electricity prices, compounded by the constraints of limited charging infrastructure. This paper tackles these complexities by leveraging Deep Reinforcement Learning (DRL), which excels in learning directly from environmental interactions without relying on pre-defined models. We conceive two augmented Markov Decision Processes (MDPs) and propose a novel Hierarchical Deep Reinforcement Learning (HDRL) algorithm called Double Actor-Critic Multi-Agent Proximal Policy Optimization (DAC-MAPPO). The proposed algorithm enhances learning efficiency and convergence speed by integrating the MAPPO algorithm into the DAC architecture. Specifically, a centralized high-level agent is responsible for making charger allocation decisions, while multiple decentralized low-level agents determine the charging power for each EB at every time step. Experimental evaluations using real-world data demonstrate the superior performance and effectiveness of the DAC-MAPPO algorithm.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Hierarchical Deep Reinforcement Learning for Charging Scheduling of Electric Buses with Uncertainties


    Contributors:
    Qi, Jiaju (author) / Lei, Lei (author) / Jonsson, Thorsteinn (author)


    Publication date :

    2024-09-24


    Size :

    754247 byte





    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English




    CHARGING SCHEDULING SYSTEMS AND METHODS THEREOF FOR ELECTRIC BUSES

    WANG JOHN C / HOU YI-AN / KUNG CHUN-HUNG et al. | European Patent Office | 2022

    Free access

    Charging scheduling systems and methods thereof for electric buses

    WANG JOHN C / HOU YI-AN / KUNG CHUN-HUNG et al. | European Patent Office | 2023

    Free access

    Charging system for electric buses

    SHIN DAE SEOB | European Patent Office | 2020

    Free access

    Operation and Scheduling of Pure Electric Buses under Regular Charging Mode

    Jiang, Mengyan / Zhang, Yi / Zhang, Canrong et al. | IEEE | 2018