Driving at night is more challenging and dangerous than driving during the day. Modeling driver eye movement and attention allocation during night driving can help guide unmanned intelligent vehicles and improve safety during similar situations. However, until now, few studies have modeled a drivers’ true fixations and attention allocation in specific night circumstance. Therefore, we collected an eye tracking dataset from 30 experienced drivers while they viewed night driving videos under a hypothetical driving condition, termed Driver Fixation Dataset in night (DrFixD(night)). Based on DrFixD(night) which includes multiple drivers’ attention allocation, we proposed a spatio-temporal dual-encoder network model, named as STDE-Net, to improve saliency detection in night driving condition. The model includes three modules: i) spatio-temporal dual encoding module, ii) fusion module based on attention mechanism, and iii) decoding module. A convolutional LSTM is employed to learn the time connection of video sequences, and a convolution neural network combined pyramid dilated convolution is adopted to extract spatial features in the spatio-temporal dual encoding module. The attention mechanism is exploited to fuse the temporal and spatial features together and selectively highlight the significant features in night traffic scene. We compared the proposed model with other traditional methods and deep learning models, both qualitatively and quantitatively, and found that the proposed model can predict driver’s fixation more accurately. Specifically, the proposed model not only predicts the main goals, but also predicts the important sub goals, such as pedestrians, bicycles and so on, showing excellent prediction of dimly lit targets at night.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Driving Visual Saliency Prediction of Dynamic Night Scenes via a Spatio-Temporal Dual-Encoder Network


    Contributors:
    Deng, Tao (author) / Jiang, Lianfang (author) / Shi, Yi (author) / Wu, Jiang (author) / Wu, Zhangbi (author) / Yan, Shun (author) / Zhang, Xianshi (author) / Yan, Hongmei (author)


    Publication date :

    2024-03-01


    Size :

    3141019 byte




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English



    Pothole detection using spatio‐temporal saliency

    Jang, Dong‐Won / Park, Rae‐Hong | Wiley | 2016

    Free access

    Pothole detection using spatio-temporal saliency

    Jang, Dong-Won / Park, Rae-Hong | IET | 2016

    Free access

    Online hash tracking with spatio-temporal saliency auxiliary

    Fang, Jianwu / Xu, Hongke / Wang, Qi et al. | British Library Online Contents | 2017


    Global spatio‐temporal dynamic capturing network‐based traffic flow prediction

    Haoran Sun / Yanling Wei / Xueliang Huang et al. | DOAJ | 2023

    Free access

    Global spatio‐temporal dynamic capturing network‐based traffic flow prediction

    Sun, Haoran / Wei, Yanling / Huang, Xueliang et al. | Wiley | 2023

    Free access