Human drivers instinctively reason with commonsense knowledge to predict hazards in unfamiliar scenarios and to understand the intentions of other road users. However, this essential capability is entirely missing from traditional decision-making systems in autonomous driving. In response, this paper presents DriveLLM, a decision-making framework that integrates large language models (LLMs) with existing autonomous driving stacks. This integration allows for commonsense reasoning in decision-making. DriveLLM also features a unique cyber-physical feedback system, allowing it to learn and improve from its mistakes. In real-world case studies, the proposed framework outperforms traditional decision-making methods in complex scenarios, including difficult edge cases. Furthermore, we propose a novel approach that allows the decision-making system to interact with human inputs while guarding against adversarial attacks. Empirical evaluations demonstrate that this framework responds correctly to complex human instructions.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    DriveLLM: Charting the Path Toward Full Autonomous Driving With Large Language Models


    Contributors:
    Cui, Yaodong (author) / Huang, Shucheng (author) / Zhong, Jiaming (author) / Liu, Zhenan (author) / Wang, Yutong (author) / Sun, Chen (author) / Li, Bai (author) / Wang, Xiao (author) / Khajepour, Amir (author)

    Published in:

    Publication date :

    2024-01-01


    Size :

    8612270 byte




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English



    Personalized Autonomous Driving with Large Language Models: Field Experiments

    Cui, Can / Yang, Zichong / Zhou, Yupeng et al. | IEEE | 2024


    A Superalignment Framework in Autonomous Driving with Large Language Models

    Kong, Xiangrui / Braunl, Thomas / Fahmi, Marco et al. | IEEE | 2024


    Advancing Autonomous Driving with Large Language Models: Integration and Impact

    Ananthajothi, K / Satyaa Sudarshan, G S / Saran, J U | IEEE | 2024


    Charting the electronics path

    Buchholz, K. | Tema Archive | 1999


    Large Language Models for Human-Like Autonomous Driving: A Survey

    Li, Yun / Katsumata, Kai / Javanmardi, Ehsan et al. | IEEE | 2024