Deep learning has been widely utilized in intelligent vehicle systems, particularly in the field of driver distraction detection. However, existing methods in this application tend to focus solely on appearance or cognitive state as indicators of distraction, while neglecting the significance of temporal modeling in accurately identifying driver actions. This oversight can result in limitations such as difficulty in comprehending context, incapability to recognize gradual changes, and failure to capture complex behaviors. To address these limitations, this paper introduces a new framework based on the concept of Driver Digital Twin (DDT). The DDT framework serves as a digital replica of the driver, capturing their naturalistic driving data and behavioral models. It consists of a transformer-based driver action recognition module and a novel temporal localization module to detect distracted behaviors. Additionally, we propose a pseudo-labeled multi-task learning algorithm that includes driver emotion recognition as supplementary information for recognizing distractions. We have validated the effectiveness of our approach using three publicly available driver distraction detection benchmarks: SFDDD, AUCDD, and SynDD2. The results demonstrate that our framework achieves state-of-the-art performance in both driver action recognition and temporal localization tasks. It outperforms the leading methods by 6.5 and 0.9 percentage points on SFDDD and AUCDD, respectively. Furthermore, it ranks in the top 5% on the SynDD2 leaderboard.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Driver Digital Twin for Online Recognition of Distracted Driving Behaviors


    Contributors:
    Ma, Yunsheng (author) / Du, Runjia (author) / Abdelraouf, Amr (author) / Han, Kyungtae (author) / Gupta, Rohit (author) / Wang, Ziran (author)

    Published in:

    Publication date :

    2024-02-01


    Size :

    2861865 byte




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English



    Distracted Driver Posture Recognition

    Shankar, V. Bhanu / Thangam, S. | IEEE | 2022


    Distracted driver detection

    MADKOR AHMED / ELQATTAN YOUSSRA / ABDELHAMID ABDARHMAN S | European Patent Office | 2020

    Free access

    A Sparse Spatiotemporal Transformer for Detecting Driver Distracted Behaviors

    Nie, Linzhen / Wang, Peng / Zhai, Xukai et al. | SAE Technical Papers | 2023


    A Sparse Spatiotemporal Transformer for Detecting Driver Distracted Behaviors

    Wang, Peng / Yin, Zhishuai / Nie, Linzhen et al. | British Library Conference Proceedings | 2023