The invention discloses an automatic driving behavior integrated decision-making method based on deep reinforcement learning, and the method comprises the steps: starting from the priority and logic of two basic driving behaviors, i.e., lane changing and car following behaviors; firstly, an upper-layer deep reinforcement learning model and a lower-layer deep reinforcement learning model are adopted to process a large-scale mixed state space, composite action output of a lane changing decision and a car following decision is ensured, and a D3QN algorithm is used in an upper-layer model to distinguish potential value of an environment and value of the lane changing decision; and adopting a DDPG algorithm to process a car-following decision in the lower-layer model so as to output continuous vehicle speed control. After a mixed standard driving condition is selected to carry out coupling training on the two algorithms, the model is tested in a highly complex driving scene reconstructed by using NGSIM data. According to the integrated driving decision model provided by the invention, the collaboration of car following and lane changing decisions is improved, and the lane dominant function and a dangerous action shielding mechanism are integrated into the decision model, so that the driving speed is remarkably improved, and the integrated driving decision model is superior to other intelligent driving decision models.

    本发明公开了一种基于深度强化学习的自动驾驶行为一体化决策方法,从两种基本驾驶行为,即换道和跟驰行为的优先级和逻辑出发,首先采用上下两层深度强化学习模型处理大规模混合状态空间并保证换道决策和跟驰决策的复合动作输出,在上层模型中使用D3QN算法来区分环境的潜在价值和换道决策的价值,在下层模型采用DDPG算法处理跟驰决策从而输出连续的车速控制。选取混合标准驾驶工况对两种算法进行耦合训练之后,在利用NGSIM数据重建的高度复杂的驾驶场景中对模型进行测试。本发明提出的集成驾驶决策模型提高了跟驰与换道决策的协同性,并将车道优势函数与危险动作屏蔽机制整合到决策模型中,显著提升了行驶速度,优于其他的智能驾驶决策模型。


    Access

    Download


    Export, share and cite



    Title :

    Automatic driving behavior integrated decision-making method based on deep reinforcement learning


    Additional title:

    一种基于深度强化学习的自动驾驶行为一体化决策方法


    Contributors:
    PENG JIANKUN (author) / ZHANG SIYU (author) / WANG YUCHENG (author) / LI ZHIBIN (author) / ZHOU YANG (author) / YUAN SHENG (author)

    Publication date :

    2022-11-11


    Type of media :

    Patent


    Type of material :

    Electronic Resource


    Language :

    Chinese


    Classification :

    IPC:    B60W CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION , Gemeinsame Steuerung oder Regelung von Fahrzeug-Unteraggregaten verschiedenen Typs oder verschiedener Funktion / G06N COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS , Rechnersysteme, basierend auf spezifischen Rechenmodellen



    Automatic driving behavior decision-making method based on deep reinforcement learning

    YANG MINGZHU / LIU XIANGWEI / LI ZHUOLUO | European Patent Office | 2020

    Free access

    Automatic driving decision-making method based on deep reinforcement learning

    LIU CHENGQI / LIU SHAOWEIHUA / ZHANG YUJIE et al. | European Patent Office | 2024

    Free access

    Automatic driving decision-making system based on deep reinforcement learning

    ZHENG XIAOYAO / YAO QINGHE / ZHANG JIANPENG et al. | European Patent Office | 2024

    Free access

    Automatic driving transverse and longitudinal integrated decision-making method based on deep reinforcement learning

    IM MYOUNG-YOON / WANG JUAN / ZHOU JUNJIE et al. | European Patent Office | 2023

    Free access

    Automatic driving vehicle overtaking decision-making method based on deep reinforcement learning

    LIU JIA / LI HUIYUN / CUI YUNDUAN | European Patent Office | 2022

    Free access