Semantic segmentation is an important visual perception module of automated driving. Most of the progress has been focused on single-frame image segmentation with an optional temporal post-processing. In this paper, we propose a novel algorithm to utilize the temporal information in the semantic segmentation model using convolutional gated recurrent networks. The main motivation is to design a spatio-temporal network which can leverage motion cues for aiding segmentation and providing temporally consistent results. The proposed algorithm makes use of a fully convolutional network (FCN) that is embedded into a gated recurrent architecture. We use FCN because of its simplicity and ease of extension and the embedding can extend to other architectures. We also chose an FCN model with reasonable computational complexity suitable for real-time applications. Experimental results show consistent accuracy improvements over the baseline FCN in several datasets and it is also visually evident in our test videos shared on YouTube. The accuracy improvements for binary segmentation using F-measure were 5% and 3% in SegTrack2 and Davis respectively and the improvements for semantic segmentation in mean IoU were 5.7% and 1.7% in Synthia and Camvid respectively. To our knowledge, no prior work has been done for CNN based spatio-temporal video segmentation for automated driving and we hope that our results encourage further research in this area.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Convolutional gated recurrent networks for video semantic segmentation in automated driving


    Contributors:


    Publication date :

    2017-10-01


    Size :

    714542 byte





    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English



    Semantic Segmentation of Video Sequences with Convolutional LSTMs

    Pfeuffer, Andreas / Schulz, Karina / Dietmayer, Klaus | IEEE | 2019


    Efficient Instance and Semantic Segmentation for Automated Driving

    Petrovai, Andra / Nedevschi, Sergiu | IEEE | 2019


    EFFICIENT INSTANCE AND SEMANTIC SEGMENTATION FOR AUTOMATED DRIVING

    Petrovai, Andra / Nedevschi, Sergiu | British Library Conference Proceedings | 2019


    Automated Evaluation of Semantic Segmentation Robustness for Autonomous Driving

    Zhou, Wei / Berrio, Julie Stephany / Worrall, Stewart et al. | IEEE | 2020


    Deep semantic segmentation for automated driving: Taxonomy, roadmap and challenges

    Siam, Mennatullah / Elkerdawy, Sara / Jagersand, Martin et al. | IEEE | 2017