Human-level driving is an ultimate goal of autonomous driving. Conventional approaches formulate autonomous driving as a perception-prediction-planning framework, yet their systems do not capitalize on the inherent reasoning ability and experiential knowledge of humans. In this paper, we propose a fundamental paradigm shift from current pipelines, exploiting Large Language Models (LLMs) as a cognitive agent to integrate human-like intelligence into autonomous driving systems. Our approach, termed Agent-Driver, transforms the traditional autonomous driving pipeline by introducing a versatile tool library accessible via function calls, a cognitive memory of common sense and experiential knowledge for decision-making, and a reasoning engine capable of chain-of-thought reasoning, task planning, motion planning, and self-reflection. Powered by LLMs, our Agent-Driver is endowed with intuitive common sense and robust reasoning capabilities, thus enabling a more nuanced, human-like approach to autonomous driving. We evaluate our approach on the large-scale nuScenes benchmark, and extensive experiments substantiate that our Agent-Driver significantly outperforms the state-of-the-art driving methods by a large margin. Our approach also demonstrates superior interpretability and few-shot learning ability to these methods.


    Access

    Download


    Export, share and cite



    Title :

    A Language Agent for Autonomous Driving


    Contributors:
    Mao, Jiageng (author) / Ye, Junjie (author) / Qian, Yuxi (author) / Pavone, Marco (author) / Wang, Yue (author)

    Publication date :

    2023


    Remarks:

    COLM 2024. Project Page: https://usc-gvl.github.io/Agent-Driver/


    Type of media :

    Preprint


    Type of material :

    Electronic Resource


    Language :

    English




    A formal multi-agent language for cooperative autonomous driving scenarios

    Witsch, Andreas / Opfer, Stephan / Geihs, Kurt | IEEE | 2014


    AGENT IMPORTANCE PREDICTION FOR AUTONOMOUS DRIVING

    HAZARD CHRISTOPHER / BHAGAT AKSHAY / BUDDHARAJU BALARAMA RAJU et al. | European Patent Office | 2023

    Free access

    AGENT IMPORTANCE PREDICTION FOR AUTONOMOUS DRIVING

    HAZARD CHRISTOPHER / BHAGAT AKSHAY / BUDDHARAJU BALARAMA et al. | European Patent Office | 2023

    Free access

    AGENT IMPORTANCE PREDICTION FOR AUTONOMOUS DRIVING

    HAZARD CHRISTOPHER / BHAGAT AKSHAY / BUDDHARAJU BALARAMA RAJU et al. | European Patent Office | 2025

    Free access

    Intelligent agent importance prediction for autonomous driving

    HAZZARD CHRISTIAN / BHAGAT ASHISH / FODOLOJU BRENT R et al. | European Patent Office | 2024

    Free access