The invention relates to a curriculum reinforcement learning-based bicycle lane changing and motorcade entering control method. The method comprises the following steps of: setting reinforcement learning training curriculums according to task difficulty from simple to complex; determining a state space and an action space of reinforcement learning according to surrounding environment vehicle information and a control object obtained by a vehicle-mounted sensor, and setting a reward function in a multi-dimensional grading manner according to a vehicle state and a target task to guide a vehicle to finish chasing and lane changing to enter a motorcade; building an Actor network and a Critic network, building an environment exploration module according to a near-end strategy optimization algorithm, and updating the network by using an Adam optimizer; and selecting a corresponding reward function for training, performing a random number seed experiment on the network after each course is trained, performing a scene test on the strategy neural network obtained by final training of course learning, and checking the completion degree of the target task of the model. The problem that training is slow and difficult to converge due to sparse rewards in the process that the bicycles converge into the motorcade under the complex traffic flow situation is solved.

    本发明涉及一种基于课程强化学习的单车换道汇入车队控制方法,依据任务难度从简单到复杂设置强化学习训练课程;依据车载传感器获得的周围环境车辆信息和控制对象,确定强化学习的状态空间和动作空间,根据车辆状态和目标任务,多维分级地设置奖励函数引导车辆完成追赶、换道进入车队;搭建Actor网络、Critic网络,并依据近端策略优化算法搭建环境探索模块,利用Adam优化器更新网络;选择相应的奖励函数进行训练,对每个课程训练后的网络进行随机数种子实验,并对课程学习最终训练得到的策略神经网络进行场景测试,检验该模型目标任务的完成度。本发明解决复杂交通流情境下单车汇入车队过程中稀疏奖励造成的训练慢难收敛问题。


    Access

    Download


    Export, share and cite



    Title :

    Method for controlling bikes to enter motorcade by changing lanes based on curriculum reinforcement learning


    Additional title:

    一种基于课程强化学习的单车换道汇入车队控制方法


    Contributors:
    LIU HONGFEI (author) / YANG XIAO (author) / WAN JINTAO (author) / ZHAO ZIRUI (author)

    Publication date :

    2023-08-04


    Type of media :

    Patent


    Type of material :

    Electronic Resource


    Language :

    Chinese


    Classification :

    IPC:    B60W CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION , Gemeinsame Steuerung oder Regelung von Fahrzeug-Unteraggregaten verschiedenen Typs oder verschiedener Funktion



    Motorcade collaborative driving lane changing control method

    MA YULIN / XU YANG / HUANG ZICHAO et al. | European Patent Office | 2020

    Free access

    Motorcade management control system and motorcade management control method

    LIN GUANLUN | European Patent Office | 2020

    Free access

    Computing unloading and power distribution method in motorcade network based on deep reinforcement learning

    WANG LEI / HONG XINTAO / LIANG HONGBIN | European Patent Office | 2024

    Free access

    Collaborative lane changing decision-making method for intelligent network-connected motorcade

    ZHANG XIANHONG / HU JIA / XU ZHENGHAO et al. | European Patent Office | 2023

    Free access

    Motorcade control method based on automatic driving

    BI SHANSHAN / FENG SONG | European Patent Office | 2023

    Free access