Highlights A custom visual LLM for Distracted Driving Classification. DDLM enhances precision by integrating whole-body pose estimation for keypoint analysis. Reasoning chain in DDLM boosts logical analysis and inferential abilities. DDLM uses cost-effective zero-shot/few-shot learning, reducing training resources. Fine-tuning LLM with LoRA using specific data crucial for optimal performance.

    Abstract Driver behavior is a critical factor in driving safety, making the development of sophisticated distraction classification methods essential. Our study presents a Distracted Driving Classification (DDC) approach utilizing a visual Large Language Model (LLM), named the Distracted Driving Language Model (DDLM). The DDLM introduces whole-body human pose estimation to isolate and analyze key postural features—head, right hand, and left hand—for precise behavior classification and better interpretability. Recognizing the inherent limitations of LLMs, particularly their lack of logical reasoning abilities, we have integrated a reasoning chain framework within the DDLM, allowing it to generate clear, reasoned explanations for its assessments. Tailored specifically with relevant data, the DDLM demonstrates enhanced performance, providing detailed, context-aware evaluations of driver behaviors and corresponding risk levels. Notably outperforming standard models in both zero-shot and few-shot learning scenarios, as evidenced by tests on the 100-Driver dataset, the DDLM stands out as an advanced tool that promises significant contributions to driving safety by accurately detecting and analyzing driving distractions.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Integrating visual large language model and reasoning chain for driver behavior analysis and risk assessment


    Contributors:
    Zhang, Kunpeng (author) / Wang, Shipu (author) / Jia, Ning (author) / Zhao, Liang (author) / Han, Chunyang (author) / Li, Li (author)


    Publication date :

    2024-02-03




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English




    Driver behavior risk assessment and pedestrian awareness

    AGARWAL NAKUL / CHEN YI-TING | European Patent Office | 2023

    Free access

    Driver behavior risk assessment and pedestrian awareness

    AGARWAL NAKUL / CHEN YITING | European Patent Office | 2022

    Free access

    DRIVER BEHAVIOR RISK ASSESSMENT AND PEDESTRIAN AWARENESS

    AGARWAL NAKUL / CHEN YI-TING | European Patent Office | 2022

    Free access

    DRIVER BEHAVIOR RISK ASSESSMENT AND PEDESTRIAN AWARENESS

    AGARWAL NAKUL / CHEN YI-TING | European Patent Office | 2023

    Free access

    Visual Behaviour Analysis and Driver Cognitive Model

    Baujon, J. / Basset, M. / Gissinger, G. L. | British Library Conference Proceedings | 2001