In recent years, researchers have focused on identifying tasks related to driver state, traffic environment, and others to enhance the safety of autonomous driving assistance systems. However, current research on these tasks is conducted independently, neglecting the interconnections between the driver, traffic environment, and vehicle. In this paper, we propose a Unified Multi-task Assistive Driving Network Based on Multimodal Fusion (UMD-Net), the first unified model capable of recognizing four tasks simultaneously by utilizing multimodal data: driver behavior recognition, driver emotion recognition, traffic context recognition, and vehicle behavior recognition. In order to better enhance the synergistic effects between multiple tasks, we designed the position-sensitive multi-directional attention feature extraction subnetwork and recursive dynamic feature fusion module. The former captures the key features of multi-view images by different directions of attention mechanism to improve the generalization of the model across multiple tasks. The latter dynamically adjusts the fusion weight according to the multimodal features to enhance the representation ability of important features in multi-task learning. Our model was evaluated on the public dataset AIDE, achieving the best performance across all four tasks and a high accuracy of 95.31% in the traffic context recognition task, demonstrating the superiority of our approach. The code is available on https://github.com/Wenzhuo-Liu/UMD-Net.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    UMD-Net: A Unified Multi-Task Assistive Driving Network Based on Multimodal Fusion


    Beteiligte:
    Liu, Wenzhuo (Autor:in) / Qiao, Yicheng (Autor:in) / Li, Zhiwei (Autor:in) / Wang, Wenshuo (Autor:in) / Zhang, Wei (Autor:in) / Zhu, Jiayin (Autor:in) / Jiang, Yanhuan (Autor:in) / Wang, Li (Autor:in) / Wang, Hong (Autor:in) / Liu, Huaping (Autor:in) ... [mehr]

    Erschienen in:

    Erscheinungsdatum :

    01.08.2025


    Format / Umfang :

    3560785 byte




    Medientyp :

    Aufsatz (Zeitschrift)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch



    Safety Driving Assistive Device and Assistive method

    KANG SEJIN / KIM JINWOOK / KIM JAEHYUNG et al. | Europäisches Patentamt | 2019

    Freier Zugriff

    Assistive driving system for a wheelchair and method for controlling assistive driving system

    RICHTER MARK / HEMKENS BENJAMIN PHILLIP / EMFINGER WILLIAM ALEXANDER et al. | Europäisches Patentamt | 2019

    Freier Zugriff

    Assistive driving system for a wheelchair and method for controlling assistive driving system

    RICHTER MARK / HEMKENS BENJAMIN PHILLIP / EMFINGER WILLIAM ALEXANDER et al. | Europäisches Patentamt | 2021

    Freier Zugriff

    ASSISTIVE DRIVING SYSTEM FOR A WHEELCHAIR AND METHOD FOR CONTROLLING ASSISTIVE DRIVING SYSTEM

    RICHTER MARK / HEMKENS BENJAMIN PHILLIP / EMFINGER WILLIAM ALEXANDER et al. | Europäisches Patentamt | 2019

    Freier Zugriff

    Sparse U-PDP: A Unified Multi-Task Framework for Panoptic Driving Perception

    Wang, Hai / Qiu, Meng / Cai, Yingfeng et al. | IEEE | 2023