Reinforcement learning (RL) shows promise for autonomous driving decision-making. However, designing appropriate reward functions to guide RL agents towards complex optimization objectives is challenging. This article proposes a framework that learns the reward function from human driving data to guide RL agent's learning. The proposed framework consists of three components: trajectory sample, offline preference learning, and RL. Firstly, feasible trajectories are generated by sampling end targets from a reachable state space. Subsequently, a novel offline preference learning framework is utilized to train a transformer network by comparing generated feasible trajectories with human driving trajectories. The transformer network is used to model the human driving decision-making process, thereby obtaining a reward function. Finally, to obtain the final driving decision network, the derived reward function is incorporated into a RL framework. To validate the proposed method, a highway simulator is established where the surrounding vehicle trajectories are derived from real-world driving scenarios. Compared with baseline algorithms, the proposed method achieves the best performances in terms of decision safety and human-likeness. Additionally, the learned policy network performs well in driving decision-making tasks with longer total decision steps. Experimental results demonstrate that the proposed method can obviate the requirement for manual design of sophisticated reward functions in RL-based autonomous driving decision-making systems.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    HGRL: Human-Driving-Data Guided Reinforcement Learning for Autonomous Driving


    Contributors:
    Zhuang, Hejian (author) / Chu, Hongqing (author) / Wang, Yafei (author) / Gao, Bingzhao (author) / Chen, Hong (author)

    Published in:

    Publication date :

    2024-12-01


    Size :

    3243451 byte




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English



    Safe Reinforcement Learning with Policy-Guided Planning for Autonomous Driving

    Rong, Jikun / Luan, Nan | British Library Conference Proceedings | 2020


    Offline Reinforcement Learning for Autonomous Driving with Real World Driving Data

    Fang, Xing / Zhang, Qichao / Gao, Yinfeng et al. | IEEE | 2022


    Autonomous Driving with Deep Reinforcement Learning

    Zhu, Yuhua / Technische Universität Dresden | SLUB | 2023


    Robust Driving Policy Learning with Guided Meta Reinforcement Learning

    Lee, Kanghoon / Li, Jiachen / Isele, David et al. | IEEE | 2023


    Human-Guided Continual Learning for Personalized Decision-Making of Autonomous Driving

    Yang, Haohan / Zhou, Yanxin / Wu, Jingda et al. | IEEE | 2025