Driver fatigue detection is one of the crucial methods to ensure driving safety. This paper proposes a driver fatigue detection model through multimodal fusion based on convolutional neural networks (CNN) and gated transformer networks (GTN). Specifically, the driver’s facial features are extracted using CNN and combined with the driver’s electrocardiogram (ECG) and vehicle state features to form multivariate time series. Then, the GTN employs self-attention and masking mechanisms to capture the channel-wise and step-wise correlations of the input, fully leveraging the advantages of complementarity among different modalities. To verify the validity of the proposed model, extensive experiments are conducted on our dataset collected from 15 participants using a driving simulator. Experimental results demonstrate that the fusion of three modalities achieves higher accuracy than a single modality. The trimodal model achieved an average prediction accuracy of 90% across the four fatigue levels. Therefore, the multimodal-based driver fatigue detection method is feasible for autonomous vehicles (AVs).


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    Multimodal Features Fusion for Driver Fatigue Detection Based on CNN-GTN Learning


    Beteiligte:
    Zhao, Shuaijie (Autor:in) / Du, Aimin (Autor:in) / Han, Yeyang (Autor:in) / Yi, Jiwei (Autor:in) / Zhu, Zhongpan (Autor:in)


    Erscheinungsdatum :

    2023-10-27


    Format / Umfang :

    4222712 byte




    Medientyp :

    Aufsatz (Konferenz)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch



    Multimodal Features for Detection of Driver Stress and Fatigue: Review

    Nemcova, Andrea / Svozilova, Veronika / Bucsuhazy, Katerina et al. | IEEE | 2021


    Multimodal Detection of Driver Distraction

    M. Eskenazi / L. Morency / A. W. Black et al. | NTIS | 2017


    Driver Drowsiness Behavior Detection and Analysis Using Vision-Based Multimodal Features for Driving Safety

    Gopinath, Aditya / Li, Bing / Brand, Howard et al. | SAE Technical Papers | 2020


    Driver Drowsiness Behavior Detection and Analysis Using Vision-Based Multimodal Features for Driving Safety

    Li, Rui / Brand, Howard / Gopinath, Aditya et al. | British Library Conference Proceedings | 2020


    Driver Drowsiness Behavior Detection and Analysis Using Vision-Based Multimodal Features for Driving Safety

    Li, Rui / Brand, Howard / Gopinath, Aditya et al. | British Library Conference Proceedings | 2020