Over the last year, significant advancements have been made in the realms of large language models (LLMs) and multi-modal large language models (MLLMs), particularly in their application to autonomous driving. These models have showcased remarkable abilities in processing and interacting with complex information. In autonomous driving, LLMs and MLLMs are extensively used, requiring access to sensitive vehicle data such as precise locations, images, and road conditions. This data is transmitted to an LLM-based inference cloud for advanced analysis. However, concerns arise regarding data security, as the protection against data and privacy breaches primarily depends on the LLM’s inherent security measures, without additional scrutiny or evaluation of the LLM’s inference outputs. Despite its importance, the security aspect of LLMs in autonomous driving remains underexplored. Addressing this gap, our research introduces a novel security framework for autonomous vehicles, utilizing a multi-agent LLM approach. This framework is designed to safeguard sensitive information associated with autonomous vehicles from potential leaks, while also ensuring that LLM outputs adhere to driving regulations and align with human values. It includes mechanisms to filter out irrelevant queries and verify the safety and reliability of LLM outputs. Utilizing this framework, we evaluated the security, privacy, and cost aspects of eleven large language model-driven autonomous driving cues. Additionally, we performed QA tests on these driving prompts, which successfully demonstrated the framework’s efficacy.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    A Superalignment Framework in Autonomous Driving with Large Language Models


    Contributors:
    Kong, Xiangrui (author) / Braunl, Thomas (author) / Fahmi, Marco (author) / Wang, Yue (author)


    Publication date :

    2024-06-02


    Size :

    1060761 byte





    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English



    Personalized Autonomous Driving with Large Language Models: Field Experiments

    Cui, Can / Yang, Zichong / Zhou, Yupeng et al. | IEEE | 2024


    Advancing Autonomous Driving with Large Language Models: Integration and Impact

    Ananthajothi, K / Satyaa Sudarshan, G S / Saran, J U | IEEE | 2024


    Large Language Models for Human-Like Autonomous Driving: A Survey

    Li, Yun / Katsumata, Kai / Javanmardi, Ehsan et al. | IEEE | 2024


    DriveLLM: Charting the Path Toward Full Autonomous Driving With Large Language Models

    Cui, Yaodong / Huang, Shucheng / Zhong, Jiaming et al. | IEEE | 2024