Precise scene understanding based on perception sensors' data is important for assisted and automated driving (AAD) functions, to enable accurate decision-making processes and safe navigation. Among various perception tasks using camera images (e.g. object detection, semantic segmentation), panoptic segmentation shows promising scene understanding capability in terms of recognizing and classifying different types and objects, imminent obstacles, and drivable space at a pixel level. While current panoptic segmentation methods exhibit good potential for AAD perception under ‘ideal’ conditions, there are no systematic studies investigating the effects that various degradation factors can have on the quality of the data generated by automotive cameras. Therefore, in this paper, we consider 5 categories of camera data degradation models, namely light level, adverse weather, internal sensor noises, motion blur and compression artefacts. These 5 categories include 11 potential degradation models, with different degradation levels. Based on these 11 models and multiple degradation levels, we synthesize an augmented version of Cityscape, named the Degraded-Cityscapes (D-Cityscapes). Moreover, for the environmental light level, we propose a new synthetic method with generative adversarial learning and zero-reference deep curve estimation to simulate 3 degraded light levels including low light, night light, and extreme light. To compare the effect of the implemented camera degradation factors, we run extensive tests using a panoptic segmentation network (i.e. EfficientPS), quantifying how the performance metrics vary when the data are degraded. Based on the evaluation results, we demonstrate that extreme snow, blur, and light are the most threatening conditions for panoptic segmentation in AAD, while EfficientPS can cope well with light fog, compression, and blur, which can provide insights for future research directions.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    The Effect of Camera Data Degradation Factors on Panoptic Segmentation for Automated Driving


    Contributors:


    Publication date :

    2023-09-24


    Size :

    657485 byte





    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English



    Multi-task Network for Panoptic Segmentation in Automated Driving

    Petrovai, Andra / Nedevschi, Sergiu | IEEE | 2019


    Panoptic Segmentation from Stitched Panoramic View for Automated Driving

    Kinzig, Christian / Miller, Henning / Lauer, Martin et al. | IEEE | 2024


    REAL-TIME PANOPTIC SEGMENTATION WITH PROTOTYPE MASKS FOR AUTOMATED DRIVING

    Petrovai, Andra / Nedevschi, Sergiu | British Library Conference Proceedings | 2020



    Depth-Aware Panoptic Segmentation with Mask Transformers and Panoptic Bins for Autonomous Driving

    Petrovai, Andra / Miclea, Vlad-Cristian / Nedevschi, Sergiu | IEEE | 2024