Recent representative scene parsing methods based on Convolutional Neural Networks (CNN) have greatly improved spatial resolution of pixel-wise labelling by exploiting multi-scale features and refined boundaries. However, the vast majority of previous works only utilize the color or textural information of images, without considering the depth information, which is beneficial for semantic reasoning. In this paper, we take advantages of the mutual benefit and strong correlation between depth information and semantic information in scene parsing by introducing the Built-in Depth-Semantic Coupled Encoding (BDSCE) module, which adaptively fuses RGB and depth features, and selectively highlights the depth-discriminative features. The proposed BDSCE module is compatible with existing CNN based methods, and can greatly improve scene parsing performance, particularly in those categories that have clear depth distinction and might be misclassified with RGB-only features. Furthermore, we also extend our proposed module to other urban scene semantic reasoning tasks such as vehicle detection and road segmentation, which are implemented by effectively learning and exploiting the encoded depth-semantics and transferring the learned representations with fine-tuning. The extensive experiments on the popular datasets Cityscapes and KITTI demonstrate that our method performs quite well and can significantly improve the state-of-the-art methods.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Built-in Depth-Semantic Coupled Encoding for Scene Parsing, Vehicle Detection, and Road Segmentation


    Contributors:
    Liu, Shiyu (author) / Zhang, Haofeng (author) / Shao, Ling (author) / Yang, Jingyu (author)


    Publication date :

    2021-09-01


    Size :

    11741398 byte




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English



    Spatial Prior for Nonparametric Road Scene Parsing

    Di, Shuai / Zhang, Honggang / Mei, Xue et al. | IEEE | 2015



    Exploiting Large Image Sets for Road Scene Parsing

    Alvarez, Jose M | Online Contents | 2016


    Exploiting Large Image Sets for Road Scene Parsing

    Alvarez, Jose M. / Salzmann, Mathieu / Barnes, Nick | IEEE | 2016


    Nighttime Road Scene Parsing by Unsupervised Domain Adaptation

    Song, Can / Wu, Jin / Zhu, Lei et al. | IEEE | 2022