In this paper, we tackle the newly introduced panoptic segmentation task. Panoptic segmentation unifies semantic and instance segmentation and leverages the capabilities of these complementary tasks by providing pixel and instance level classification. Current state-of-the-art approaches employ either separate networks for each task or a single network for both task and post processing heuristics fuse the outputs into the final panoptic segmentation. Instead, our approach solves all three tasks including panoptic segmentation with an end-to-end learnable fully convolutional neural network. We build upon the Mask R-CNN framework with a shared backbone and individual network heads for each task. Our semantic segmentation head uses multi-scale information from the Feature Pyramid Network, while the panoptic head learns to fuse the semantic segmentation logits with variable number of instance segmentation logits. Moreover, the panoptic head refines the outputs of the network, improving the semantic segmentation results. Experimental results on the challenging Cityscapes dataset demonstrate that the proposed solution achieves significant improvements for both panoptic segmentation and semantic segmentation.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    Multi-task Network for Panoptic Segmentation in Automated Driving


    Beteiligte:


    Erscheinungsdatum :

    2019-10-01


    Format / Umfang :

    2859039 byte




    Medientyp :

    Aufsatz (Konferenz)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch




    The Effect of Camera Data Degradation Factors on Panoptic Segmentation for Automated Driving

    Wang, Yiting / Zhao, Haonan / Debattista, Kurt et al. | IEEE | 2023


    Sparse U-PDP: A Unified Multi-Task Framework for Panoptic Driving Perception

    Wang, Hai / Qiu, Meng / Cai, Yingfeng et al. | IEEE | 2023


    Location-Guided LiDAR-Based Panoptic Segmentation for Autonomous Driving

    Xian, Guozeng / Ji, Changyun / Zhou, Lin et al. | IEEE | 2023


    Single Network Panoptic Segmentation for Street Scene Understanding

    de Geus, Daan / Meletis, Panagiotis / Dubbelman, Gijs | IEEE | 2019