In recent years, researchers have focused on identifying tasks related to driver state, traffic environment, and others to enhance the safety of autonomous driving assistance systems. However, current research on these tasks is conducted independently, neglecting the interconnections between the driver, traffic environment, and vehicle. In this paper, we propose a Unified Multi-task Assistive Driving Network Based on Multimodal Fusion (UMD-Net), the first unified model capable of recognizing four tasks simultaneously by utilizing multimodal data: driver behavior recognition, driver emotion recognition, traffic context recognition, and vehicle behavior recognition. In order to better enhance the synergistic effects between multiple tasks, we designed the position-sensitive multi-directional attention feature extraction subnetwork and recursive dynamic feature fusion module. The former captures the key features of multi-view images by different directions of attention mechanism to improve the generalization of the model across multiple tasks. The latter dynamically adjusts the fusion weight according to the multimodal features to enhance the representation ability of important features in multi-task learning. Our model was evaluated on the public dataset AIDE, achieving the best performance across all four tasks and a high accuracy of 95.31% in the traffic context recognition task, demonstrating the superiority of our approach. The code is available on https://github.com/Wenzhuo-Liu/UMD-Net.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    UMD-Net: A Unified Multi-Task Assistive Driving Network Based on Multimodal Fusion


    Contributors:
    Liu, Wenzhuo (author) / Qiao, Yicheng (author) / Li, Zhiwei (author) / Wang, Wenshuo (author) / Zhang, Wei (author) / Zhu, Jiayin (author) / Jiang, Yanhuan (author) / Wang, Li (author) / Wang, Hong (author) / Liu, Huaping (author) ... [more]


    Publication date :

    2025-08-01


    Size :

    3560785 byte




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English



    Safety Driving Assistive Device and Assistive method

    KANG SEJIN / KIM JINWOOK / KIM JAEHYUNG et al. | European Patent Office | 2019

    Free access

    Assistive driving system for a wheelchair and method for controlling assistive driving system

    RICHTER MARK / HEMKENS BENJAMIN PHILLIP / EMFINGER WILLIAM ALEXANDER et al. | European Patent Office | 2019

    Free access

    Assistive driving system for a wheelchair and method for controlling assistive driving system

    RICHTER MARK / HEMKENS BENJAMIN PHILLIP / EMFINGER WILLIAM ALEXANDER et al. | European Patent Office | 2021

    Free access

    ASSISTIVE DRIVING SYSTEM FOR A WHEELCHAIR AND METHOD FOR CONTROLLING ASSISTIVE DRIVING SYSTEM

    RICHTER MARK / HEMKENS BENJAMIN PHILLIP / EMFINGER WILLIAM ALEXANDER et al. | European Patent Office | 2019

    Free access

    Sparse U-PDP: A Unified Multi-Task Framework for Panoptic Driving Perception

    Wang, Hai / Qiu, Meng / Cai, Yingfeng et al. | IEEE | 2023