Semantic segmentation and depth estimation play important roles in the field of autonomous driving. In recent years, the advantages of Convolutional Neural Networks (CNNs) have allowed these two topics to flourish. However, people always solve these two tasks separately and rarely solve them in a united model. In this paper, we propose a Mutual Encouragement Network (MENet), which includes a semantic segmentation branch and a disparity regression branch, and simultaneously generates semantic map and visual disparity. In the cost volume construction phase, the depth information is embedded in the semantic segmentation branch to increase contextual understanding. Similarly, the semantic information is also included in the disparity regression branch to generate more accurate disparity. Two branches mutually promote each other during training phase and inference phase. We conducted our method on the popular dataset KITTI, and the experimental results show that our method can outperform the state-of-the-art methods on both visual disparity generation and semantic segmentation. In addition, extensive ablation studies also demonstrate that the two tasks in our method can facilitate each other significantly with the proposed approach.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    When Visual Disparity Generation Meets Semantic Segmentation: A Mutual Encouragement Approach


    Contributors:
    Zhang, Xiaohong (author) / Chen, Yi (author) / Zhang, Haofeng (author) / Wang, Shuihua (author) / Lu, Jianfeng (author) / Yang, Jingyu (author)


    Publication date :

    2021-03-01


    Size :

    6569735 byte




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English




    Disparity weighted loss for semantic segmentation of driving scenes

    Loukkal, Abdelhak / Grandvalet, Yves / Li, You | IEEE | 2019


    When wheel meets rail

    Tuzik, Robert E. | IuD Bahn | 2000


    When Discrete Meets Differential

    Lin, W. Y. / Tan, G. C. / Cheong, L. F. | British Library Online Contents | 2010


    When multiplicity meets productivity

    Weart, Walter | IuD Bahn | 2013