In mixed autonomy traffic environment, every decision made by an autonomous-driving car may have a great impact on the transportation system. Because of the complex interaction between vehicles, it is challenging to make decisions that can ensure both high traffic efficiency and safety now and futher. Connected automated vehicles (CAVs) have great potential to improve the quality of decision-making in this continuous, highly dynamic and interactive environment because of their stronger sensing and communicating ability. For multi-vehicle collaborative decision-making algorithms based on deep reinforcement learning (DRL), we need to represent the interactions between vehicles to obtain interactive features. The representation in this aspect directly affects the learning efficiency and the quality of the learned policy. To this end, we propose a CAV decision-making architecture based on transformer and reinforcement learning algorithms. A learnable policy token is used as the learning medium of the multi-vehicle joint policy, the states of all vehicles in the area of interest can be adaptively noticed in order to extract interactive features among agents. We also design an intuitive physical positional encodings, the redundant location information of which optimizes the performance of the network. Simulations show that our model can make good use of all the state information of vehicles in traffic scenario, so as to obtain high-quality driving decisions that meet efficiency and safety objectives. The comparison shows that our method significantly improves existing DRL-based multi-vehicle cooperative decision-making algorithms.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    SPformer: A Transformer Based DRL Decision Making Method for Connected Automated Vehicles


    Contributors:
    Han, Ye (author) / Zhang, Lijun (author) / Meng, Dejian (author) / Hu, Xingyu (author) / Lu, Yixia (author)


    Publication date :

    2024-09-24


    Size :

    1927647 byte





    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English



    Cooperative decision making for connected automated vehicles in multiple driving scenarios

    Wang, Jinzhu / Ma, Zhixiong / Zhu, Xichan et al. | Wiley | 2023

    Free access

    Decision Making for Connected and Automated Vehicles: A Max-Plus Approach

    Fusco, Mauro / Semsar-Kazerooni, Elham / Zegers, Jeroen C. et al. | IEEE | 2018


    Guest editorial: Decision making and control for connected and automated vehicles

    Chen Lv / Peng Hang / Yang Xing et al. | DOAJ | 2022

    Free access

    Guest editorial: Decision making and control for connected and automated vehicles

    Lv, Chen / Hang, Peng / Xing, Yang et al. | Wiley | 2022

    Free access

    Cooperative decision making for connected automated vehicles in multiple driving scenarios

    Jinzhu Wang / Zhixiong Ma / Xichan Zhu et al. | DOAJ | 2023

    Free access