Typical reinforcement learning (RL) requires a huge amount of data before achieving an acceptable result, and its performance can be rather poor during initial interacting process. Sample inefficiency and cold-start phenomenon of RL limits its feasibility in a range of real-world applications such as traffic signal control (TSC). On the other hand, a large amount of data on TSC can be accumulated by various model-based controllers (MBCs) rooted in traffic engineering. In this context, we propose a new RL approach which can avoid the appearance of cold starts by taking advantage of MBC experiences. First, three frameworks of joint utilization of RL and MBC are summarized for TSC, and staged framework is considered to have the edge over the other two. Then, a staged noisy-net prioritized dueling double deep Q-network (NPDD-DQN) is described in detail for TSC, where MBC experiences are used in both pre-training and online training processes. Experimental evaluation demonstrates that staged NPDD-DQN can achieve a boost in initial performance as compared to pure NPDD-DQN that does not utilize any control experiences, and learn to improve final performance beyond the underlying MBC. The effectiveness of the proposed method opens up the possibility of real implementation of RL in TSC.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    A cold-start-free reinforcement learning approach for traffic signal control


    Contributors:
    Xiao, Nan (author) / Yu, Liang (author) / Yu, Jinqiang (author) / Chen, Peng (author) / Liu, Yuehu (author)

    Published in:

    Publication date :

    2022-06-15


    Size :

    10 pages




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    Unknown




    A Deep Reinforcement Learning Approach to Traffic Signal Control

    Razack, Aquib Junaid / Ajith, Vysyakh / Gupta, Rajiv | IEEE | 2021


    A Deep Reinforcement Learning Approach for Fair Traffic Signal Control

    Raeis, Majid / Leon-Garcia, Alberto | IEEE | 2021


    Agent-Based Traffic Signal Control Using a Reinforcement Learning Approach

    Agafonov, Anton / Yumaganov, Alexander | IEEE | 2021


    Deep Reinforcement Learning-based Traffic Signal Control

    Ruan, Junyun / Tang, Jinzhuo / Gao, Ge et al. | IEEE | 2023