The worldwide popularity of electric automated guided vehicles and autonomous vehicles in container terminals requires efficient vehicle scheduling management considering the capacitated battery energy and the charging station limitation. In this paper, the recharging-considered vehicle scheduling problem is formulated as a decentralized partially observable Markov decision process to maximize the cumulative reward, providing a highly adaptive decision-making mechanism for multi-agent-powered terminal transport system by supporting decentralized decisions and accommodating partial observability. Considering the limited number of charging stations and tight schedules, a novel scheduling method based on the actor-critic multi-agent deep reinforcement learning framework is developed to facilitate cooperation among vehicles and charging stations and enhance the stability and efficiency of the learning process. Furthermore, to address the challenge on algorithm convergence due to the vast state space, we employ a heterogeneous graph neural network in the proposed framework for feature extraction and a multi-agent proximal policy optimization algorithm for parameter training. Numerical results indicate that the proposed method outperforms the distributed-agent deep reinforcement learning and several benchmark heuristics, showcasing its superior performance in both solution quality and efficiency. Moreover, the well-trained model can be directly applied to various scenarios, demonstrating its high generalization capability.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Multi-Agent Deep Reinforcement Learning for Recharging-Considered Vehicle Scheduling Problem in Container Terminals


    Contributors:
    Che, Ada (author) / Wang, Ziliang (author) / Zhou, Chenhao (author)

    Published in:

    Publication date :

    2024-11-01


    Size :

    10048392 byte




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English





    A method integrating simulation and reinforcement learning for operation scheduling in container terminals

    Qingcheng Zeng / Zhongzhen Yang / Xiangpei Hu | DOAJ | 2012

    Free access

    Freight train scheduling via decentralised multi-agent deep reinforcement learning

    Bretas, Allan / Mendes, Alexandre / Chalup, Stephan K. | TIBKAT | 2021

    Free access