One of the fundamental challenges in the design of perception systems for autonomous vehicles is validating the performance of each algorithm under a comprehensive variety of operating conditions. In the case of vision-based semantic segmentation, there are known issues when encountering new scenarios that are sufficiently different to the training data. In addition, even small variations in environmental conditions, such as illumination and precipitation, can affect the classification performance of the segmentation model. Given the reliance on visual information, these effects often translate into poor semantic pixel classification which can potentially lead to catastrophic consequences when driving autonomously. This paper presents a novel method for analyzing the robustness of semantic segmentation models and provides a number of metrics to evaluate the classification performance over a variety of environmental conditions. The process incorporates an additional sensor (lidar) to automate the process and improve the system integrity, eliminating the need for labor-intensive hand labeling of validation data. The experimental results are presented based on multiple datasets collected at different times of the year with different environmental conditions. We extract the ”Road” class using the lidar to demonstrate the concepts, but this could be extended to other classes with different feature detection algorithms. These results show that the semantic segmentation performance varies depending on the weather, camera parameters, and existence of shadows. The results also demonstrate how the metrics can be used to compare and validate the performance after making improvements to a model, and compare the performance of different networks.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Automated Evaluation of Semantic Segmentation Robustness for Autonomous Driving


    Contributors:


    Publication date :

    2020-05-01


    Size :

    2135157 byte




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English



    On Adversarial Robustness of Semantic Segmentation Models for Automated Driving

    Yin, Huilin / Wang, Ruining / Liu, Boyu et al. | IEEE | 2022


    Efficient Instance and Semantic Segmentation for Automated Driving

    Petrovai, Andra / Nedevschi, Sergiu | IEEE | 2019


    RGB and LiDAR fusion based 3D Semantic Segmentation for Autonomous Driving

    El Madawi, Khaled / Rashed, Hazem / El Sallab, Ahmad et al. | IEEE | 2019


    Deep semantic segmentation for automated driving: Taxonomy, roadmap and challenges

    Siam, Mennatullah / Elkerdawy, Sara / Jagersand, Martin et al. | IEEE | 2017