Methods and systems for training an autonomous driving system using a vision-language planning (VLP) model. Image data is obtained from a vehicle-mounted camera, encompassing details about agents situated within the external environment. Via image processing, the system identifies these agents within the environment. A Bird's Eye View (BEV) representation of the surroundings is then generated, encapsulating the spatiotemporal information linked to the vehicle and the recognized agents. Execution of the VLP machine learning model begins by extracting vision-based planning features from the BEV, and receiving or generating textual information characterizing various attributes of the vehicle within the environment. Text-based planning features are extracted from this textual information. To enhance model performance, a contrastive learning model is engaged to establish similarities between the vision-based and text-based planning features, and a predicted trajectory is output based on the similarities.


    Zugriff

    Download


    Exportieren, teilen und zitieren



    Titel :

    SYSTEMS AND METHODS FOR VISION-LANGUAGE PLANNING (VLP) FOUNDATION MODELS FOR AUTONOMOUS DRIVING


    Beteiligte:
    PAN CHENBIN (Autor:in) / YAMAN BURHANEDDIN (Autor:in) / NESTI TOMMASO (Autor:in) / MALLIK ABHIRUP (Autor:in) / GUO YULIANG (Autor:in) / REN LIU (Autor:in)

    Erscheinungsdatum :

    15.05.2025


    Medientyp :

    Patent


    Format :

    Elektronische Ressource


    Sprache :

    Englisch


    Klassifikation :

    IPC:    G06V / B60W CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION , Gemeinsame Steuerung oder Regelung von Fahrzeug-Unteraggregaten verschiedenen Typs oder verschiedener Funktion



    Words to Wheels: Vision-Based Autonomous Driving Understanding Human Language Instructions Using Foundation Models

    Ryu, Chanhoe / Seong, Hyunki / Lee, Daegyu et al. | ArXiv | 2024

    Freier Zugriff



    Evaluation of Safety Cognition Capability in Vision-Language Models for Autonomous Driving

    Zhang, Enming / Gong, Peizhe / Dai, Xingyuan et al. | ArXiv | 2025

    Freier Zugriff