The invention relates to the technical field of automatic driving, and discloses an automatic driving method and system based on empirical playback constraint strategy optimization, and the method comprises the steps: initializing strategy neural network parameters and value neural network parameters; designing an overhead function and a reward function, and interacting the intelligent agent with the environment to obtain a track; the empirical pool collects the trajectory and the probability distribution of the strategy neural network on the trajectory, and extracts a batch of trajectory with a specified size from the empirical pool for training to obtain training data; the current strategy is evaluated by using the value neural network, the strategy neural network is jointly updated through a strategy gradient algorithm and trust domain constraints, the value neural network is updated by means of a mean square error loss function, and if the neural network meets performance requirements, the neural network is applied to automatic driving to obtain the automatic driving method. According to the method, the RECPO algorithm is applied to expressway vehicle control, and a state space including the vehicle distance and the vehicle speed and an action space for vehicle speed adjustment and lane changing control are designed.

    本发明涉及自动驾驶技术领域,公开了基于经验回放约束策略优化的自动驾驶方法和系统,方法包括初始化策略神经网络参数和价值神经网络参数;设计开销函数和奖励函数,智能体与环境交互,获得轨迹;经验池收集轨迹和策略神经网络对轨迹的概率分布,从经验池中抽取指定大小批次的轨迹进行训练,获得训练数据;利用价值神经网络来评估当前策略,并通过策略梯度算法和信任域约束共同更新策略神经网络,借助均方误差损失函数来更新价值神经网络,如果神经网络达到性能要求,将其应用于自动驾驶,得到自动驾驶方法。本发明将提出的RECPO算法应用于高速公路车辆控制,设计了包括车辆距离和车速在内的状态空间,以及车速调整和变道控制的动作空间。


    Access

    Download


    Export, share and cite



    Title :

    Automatic driving method and system based on experience playback constraint strategy optimization


    Additional title:

    基于经验回放约束策略优化的自动驾驶方法和系统


    Contributors:
    ZHAO RUI (author) / CHEN ZIGUO (author) / FAN YUZE (author) / GAO FEI (author) / GAO ZHENHAI (author)

    Publication date :

    2024-07-30


    Type of media :

    Patent


    Type of material :

    Electronic Resource


    Language :

    Chinese


    Classification :

    IPC:    G06N COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS , Rechnersysteme, basierend auf spezifischen Rechenmodellen / B60W CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION , Gemeinsame Steuerung oder Regelung von Fahrzeug-Unteraggregaten verschiedenen Typs oder verschiedener Funktion



    Automatic driving multi-sensor synchronous playback method and system

    XIAO XIN / CHEN CHENG / ZHANG YANG | European Patent Office | 2023

    Free access

    TD3 unmanned driving decision-making method based on multi-experience playback pool

    DU YU / JIANG ANNI / ZHANG HAO et al. | European Patent Office | 2024

    Free access

    Vehicle automatic driving identification data playback method, system and device and storage medium

    HU SHENG / REN FAN / OU HIROSHI et al. | European Patent Office | 2022

    Free access

    Automatic road intersection management method based on projection type constraint strategy optimization

    ZHAO RUI / WANG KUI / GAO FEI et al. | European Patent Office | 2023

    Free access

    Driving playback method in intelligent cabin

    XIONG CHANGCHUN / WANG JINGGUI / LEE GUK-WON et al. | European Patent Office | 2022

    Free access