A ramp confluence area mixed traffic flow control method based on time-space diagram neural network reinforcement learning comprises the following steps: step 1, setting a scene of ramp confluence mixed traffic flow control, and defining matrix representation of environment and self state information which can be sensed by an intelligent agent; 2, constructing a vehicle graph network of each time step in a given time period, wherein the vehicle graph network is used for representing space and time relevance of vehicles in the scene; 3, defining an action space and a reward function of the reinforcement learning agent; based on setting of a scene, defining an action space as a mixed action space of lane changing and acceleration and deceleration, and setting a target reward function to guide an intelligent agent to learn an optimal action strategy; 4, training and testing the model; and training the model by using experience playback and a target network, and then testing the performance of the reinforcement learning agent model under different traffic conditions. According to the method, the efficient control of the CAV in the mixed traffic flow in the ramp confluence traffic bottleneck area is realized.

    一种基于时空图神经网络强化学习的匝道合流区混合交通流控制方法,包括:步骤1:设置匝道合流混合交通流控制的场景,并定义智能体所能感知到的环境和自身状态信息的矩阵表示;步骤2:在给定的时间周期中构建每个时间步的车辆图网络,用于表示场景的中车辆的空间和时间关联性;步骤3:定义强化学习智能体的动作空间和奖励函数;基于场景的设定,将动作空间定义为换道和加减速的混合动作空间,设置目标奖励函数以引导智能体学习最优的动作策略;步骤4:对模型进行训练和测试;使用经验回放和目标网络对模型进行训练,之后在不同的交通条件下中测试强化学习智能体模型的性能。本发明实现匝道合流交通瓶颈区混合交通流中CAV的高效控制。


    Access

    Download


    Export, share and cite



    Title :

    Ramp confluence area mixed traffic flow control method based on time-space diagram neural network reinforcement learning


    Additional title:

    基于时空图神经网络强化学习的匝道合流区混合交通流控制方法


    Contributors:
    XU DONGWEI (author) / QIU QINGWEI (author) / GAO GUANGYAN (author) / GUO HAIFENG (author)

    Publication date :

    2024-03-01


    Type of media :

    Patent


    Type of material :

    Electronic Resource


    Language :

    Chinese


    Classification :

    IPC:    G08G Anlagen zur Steuerung, Regelung oder Überwachung des Verkehrs , TRAFFIC CONTROL SYSTEMS / G06N COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS , Rechnersysteme, basierend auf spezifischen Rechenmodellen



    Mixed multi-ramp cooperative confluence control method based on multi-agent reinforcement learning

    YAN JUNQI / YU HONGXIN / ZHANG LIHUI et al. | European Patent Office | 2023

    Free access

    Ramp confluence edge cloud management and control method for mixed traffic flow

    LUO YUGONG / SHI JIA / LI KEQIANG et al. | European Patent Office | 2022

    Free access

    Hybrid traffic expressway ramp confluence area network connection automatic vehicle control method

    SUN DIHUA / LIU ZHONGCHENG / ZHAO MIN | European Patent Office | 2020

    Free access

    Vehicle ramp entrance confluence control method based on deep reinforcement learning

    TONG NING / LI FENGQI / WANG KUNRUI | European Patent Office | 2023

    Free access

    Flow distribution area mixed traffic flow control method based on graph neural network reinforcement learning

    XU DONGWEI / LIU PEIWEN / QIU QINGWEI | European Patent Office | 2023

    Free access