In autonomous driving, behavioral decision-making and trajectory planning remain huge challenges due to the large amount of uncertainty in environments and complex interaction relationships between the ego vehicle and other traffic participants. In this paper, we propose a novel fixed-horizon constrained reinforcement learning (RL) framework to solve decision-making and planning problems. Firstly, to introduce lane-level global navigation information into the lane state representation and avoid constant lane changes, we propose the constrained A-star algorithm, which can get the optimal path without constant lane changes. The optimality of the algorithm is also theoretically guaranteed. Then, to balance safety, comfort, and goal completion (reaching targets), we construct the planning problem as a constrained RL problem, in which the reward function is designed for goal completion, and two fixed-horizon constraints are developed for safety and comfort, respectively. Subsequently, a motion planning policy network (planner) with vectorized input is constructed. Finally, a dual ascent optimization method is proposed to train the planner network. With the advantage of being able to fully explore in the environment, the agent can learn an efficient decision-making and planning policy. In addition, benefiting from modeling the safety and comfort of the ego vehicle as constraints, the learned policy can guarantee the safety of the ego vehicle and achieve a good balance between goal completion and comfort. Experiments demonstrate that the proposed algorithm can achieve superior performance than existing rule-based, imitation learning-based, and typical RL-based methods.
Motion Planner With Fixed-Horizon Constrained Reinforcement Learning for Complex Autonomous Driving Scenarios
IEEE Transactions on Intelligent Vehicles ; 9 , 1 ; 1577-1588
2024-01-01
13303569 byte
Article (Journal)
Electronic Resource
English
LEARNING-BASED CRITIC FOR TUNING A MOTION PLANNER OF AUTONOMOUS DRIVING VEHICLE
European Patent Office | 2023
|LEARNING-BASED SPEED PLANNER FOR AUTONOMOUS DRIVING VEHICLE
European Patent Office | 2019
|LEARNING BASED SPEED PLANNER FOR AUTONOMOUS DRIVING VEHICLES
European Patent Office | 2019
|Learning based speed planner for autonomous driving vehicles
European Patent Office | 2021
|