Depth-aware panoptic segmentation reconstructs the 3D scene from a single image through the tasks of monocular depth estimation and panoptic segmentation. Recent works have designed unified networks where both tasks are formulated as per-pixel prediction. Depth is usually learned via regression while segmentation is posed as per-pixel classification. However, panoptic segmentation benchmarks have been recently dominated by methods that employ an alternative formulation and solve instance and semantic segmentation in a unified manner with mask classification. Motivated by this insight and by the recent success of hybrid classification-regression depth estimation methods based on adaptive bins, we propose a novel unified network based on Mask Transformer for depth-aware panoptic segmentation. Instead of predicting a global depth distribution per image, we leverage cross-task information sharing and equip the network with novel panoptic bins which perform depth discretization at panoptic mask-level. Consequently, the network can better understand the 3D structure of the scene at a more detailed level, leading to depth predictions that are more aligned with the visible objects. Moreover, image embeddings are efficiently shared by both tasks, leading to synergy during training and improved performance. We perform extensive experiments on the Cityscapes-DVPS and SemKITTI-DVPS datasets and demonstrate that the proposed Mask Transformer network with panoptic adaptive bins achieves state-of-the-art results and more accurate predictions than per-pixel dense prediction methods.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    Depth-Aware Panoptic Segmentation with Mask Transformers and Panoptic Bins for Autonomous Driving


    Beteiligte:


    Erscheinungsdatum :

    24.09.2024


    Format / Umfang :

    4108730 byte





    Medientyp :

    Aufsatz (Konferenz)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch



    Location-Guided LiDAR-Based Panoptic Segmentation for Autonomous Driving

    Xian, Guozeng / Ji, Changyun / Zhou, Lin et al. | IEEE | 2023


    EFFICIENT TRANSFORMER-BASED PANOPTIC SEGMENTATION

    AICH ABHISHEK / SUH YUMIN / SCHULTER SAMUEL et al. | Europäisches Patentamt | 2025

    Freier Zugriff

    EFFICIENT TRANSFORMER-BASED PANOPTIC SEGMENTATION

    AICH ABHISHEK / SUH YUMIN / SCHULTER SAMUEL et al. | Europäisches Patentamt | 2025

    Freier Zugriff

    INFRASTRUCTURE ANALYSIS USING PANOPTIC SEGMENTATION

    SCHULTER SAMUEL / GARG SPARSH | Europäisches Patentamt | 2023

    Freier Zugriff