The invention provides a personalized multi-agent reinforcement learning traffic control method based on a hyperpolicy network, and belongs to the technical field of traffic signal control. Constructing a reinforcement learning traffic signal control model, wherein the reinforcement learning traffic signal control model comprises a strategy network, a value network and a hyperstrategy network; the strategy network outputs action distribution, selects actions through probability sampling, and controls traffic signals; the value network outputs a value function; the hyper-policy network outputs a hyper-action; obtaining a joint value function through a super-action and value function dot product; the value network evaluates the strategy network through a joint value function; the reinforcement learning traffic signal control model performs model training through a loss function, including updating a value network through a mean square error; and the strategy network is updated by limiting the change range of new and old strategies through gradient clipping. According to the method, all agents share parameters, so that the training efficiency of the model is improved, and the generalization ability of the strategy is improved; the training efficiency is ensured, meanwhile, the individuation of the intersection is considered, the performance is improved, and the training cost is reduced.

    本发明提供基于超策略网络的个性化多智能体强化学习控制交通方法,属于交通信号控制技术领域。构建强化学习交通信号控制模型,包括:策略网络、价值网络、超策略网络;策略网络输出动作分布,通过概率采样选取动作,控制交通信号;价值网络输出值函数;超策略网络输出超动作;通过超动作和值函数点积获取联合值函数;价值网络通过联合值函数评价策略网络;强化学习交通信号控制模型通过损失函数进行模型训练,包括:通过均方误差更新价值网络;通过梯度裁剪限制新旧策略的变化范围更新策略网络。本发明通过所有智能体共享参数,提高了模型的训练效率,提高了策略的泛化能力;保证训练效率的同时考虑交叉口的个性化,提高性能并降低了训练成本。


    Access

    Download


    Export, share and cite



    Title :

    Personalized multi-agent reinforcement learning traffic control method based on hyperpolicy network


    Additional title:

    基于超策略网络的个性化多智能体强化学习控制交通方法


    Contributors:
    ZHANG CHENGWEI (author) / ZHOU KAILING (author) / LIU WANTING (author) / LI YIHONG (author)

    Publication date :

    2024-12-24


    Type of media :

    Patent


    Type of material :

    Electronic Resource


    Language :

    Chinese


    Classification :

    IPC:    G08G Anlagen zur Steuerung, Regelung oder Überwachung des Verkehrs , TRAFFIC CONTROL SYSTEMS




    Reinforcement learning-based multi-agent system for network traffic signal control

    Arel, I. / Liu, C. / Urbanik, T. et al. | Tema Archive | 2010


    Traffic adaptive control method based on multi-agent reinforcement learning

    HUANG HUA / ZHANG WEI / LI XIAOLONG et al. | European Patent Office | 2024

    Free access

    Research on Multi-Agent Reinforcement Learning Traffic Control

    Fu, Xinpeng / Chen, Simin / Liang, Qixian et al. | IEEE | 2023


    Traffic light signal control method based on multi-agent reinforcement learning

    ZHAO SHENGJIE / DENG HAO / CHEN ZHI | European Patent Office | 2022

    Free access