Predicting the future trajectories of dynamic traffic actors is a cornerstone task in autonomous driving. Though existing notable efforts have resulted in impressive performance improvements, a gap persists in scene cognitive and understanding of complex traffic semantics. This paper proposes Traj-LLM, the first to investigate the potential of using pre-trained Large Language Models (LLMs) without explicit prompt engineering to generate future motions from vehicular past trajectories and traffic scene semantics. Traj-LLM starts with sparse context joint encoding to dissect the agent and scene features into a form that LLMs understand. On this basis, we creatively explore LLMs' strong understanding capability to capture a spectrum of high-level scene knowledge and interactive information. To emulate the human-like lane focus cognitive function and enhance Traj-LLM's scene comprehension, we introduce lane-aware probabilistic learning powered by the Mamba module. Finally, a multi-modal Laplace decoder is designed to achieve scene-compliant predictions. Extensive experiments manifest that Traj-LLM, fueled by prior knowledge and understanding prowess of LLMs, together with lane-aware probability learning, transcends the state-of-the-art methods across most evaluation metrics. Moreover, the few-shot analysis serves to substantiate Traj-LLM's performance, as even with merely 50% of the dataset, it surpasses the majority of benchmarks relying on complete data utilization. This study explores endowing the trajectory prediction task with advanced capabilities inherent in LLMs, furnishing a more universal and adaptable solution for forecasting agent movements in a new way.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Traj-LLM: A New Exploration for Empowering Trajectory Prediction With Pre-Trained Large Language Models


    Contributors:
    Lan, Zhengxing (author) / Liu, Lingshan (author) / Fan, Bo (author) / Lv, Yisheng (author) / Ren, Yilong (author) / Cui, Zhiyong (author)

    Published in:

    Publication date :

    2025-02-01


    Size :

    4569186 byte




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English



    LG-Traj: LLM Guided Pedestrian Trajectory Prediction

    Chib, Pranav Singh / Singh, Pravendra | ArXiv | 2024

    Free access




    DISTILLATION-TRAINED MACHINE LEARNING MODELS FOR EFFICIENT TRAJECTORY PREDICTION

    YAN QIAOJING / TIAN RAN / GUAN YUN JIA et al. | European Patent Office | 2025

    Free access