Detecting dangerous traffic agents in videos captured by vehicle-mounted dashboard cameras (dashcams) is essential to ensure safe navigation in complex environments. Accident-related videos are just a minor portion of the driving-related Big Data, and the transient pre-accident process is highly dynamic and complex. Besides, risky and non-risky traffic agents can be similar in their appearance. These make risky traffic agent localization in the driving video particularly challenging. To this end, this article proposes an attention-guided multistream feature fusion network (AM-Net) to localize dangerous traffic agents from dashcam videos ahead of potential accidents. Two Gated Recurrent Unit (GRU) networks use object bounding box and optical flow features extracted from consecutive video frames to capture spatio-temporal cues for distinguishing risky traffic agents. An attention module, coupled with the GRUs, learns to identify traffic agents that are relevant to an accident. Fusing the two streams of global and object-level features, AM-Net predicts the riskiness scores of traffic agents in the video. In supporting this study, the article also introduces a new benchmark dataset called Risky Object Localization (ROL). The dataset contains spatial, temporal, and categorical annotations of the accident, object, and scene-level attributes. The proposed AM-Net achieves a promising performance of 85.59% AUC on the ROL dataset. Additionally, the AM-Net outperforms the current state-of-the-art for video anomaly detection by 3.5% AUC on the public DoTA dataset. A thorough ablation study further reveals AM-Net's merits by assessing the impact of its constituents.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    An Attention-Guided Multistream Feature Fusion Network for Early Localization of Risky Traffic Agents in Driving Videos


    Contributors:

    Published in:

    Publication date :

    2024-01-01


    Size :

    2556947 byte




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English



    Fusion-GRU: A Deep Learning Model for Future Bounding Box Prediction of Traffic Agents in Risky Driving Videos

    Monjurul Karim, Muhammad / Qin, Ruwen / Wang, Yinhai | Transportation Research Record | 2024


    Attention Characteristics of Drivers under Risky Traffic Situations

    Xu, Qian-Qian / Wan, Hua-Sen / Mo, Hai-Ping et al. | ASCE | 2018


    Multistream video fusion using local principal components analysis

    Sharma, Ravi K. / Pavel, Misha / Leen, Todd K. | SPIE | 1998


    RISKY DRIVING DETECTION DEVICE, RISKY DRIVING DETECTION SYSTEM, AND RISKY DRIVING DETECTION PROGRAM

    UEDA KENKI / TACHIBANA RYOSUKE / KAWABATA SHINICHIRO et al. | European Patent Office | 2022

    Free access

    Risky Driving or Risky Drivers?

    Leal, Nerida / Watson, Barry / Armstrong, Kerry | Transportation Research Record | 2010