Image segmentation plays a critical role in autonomous driving by providing vehicles with a detailed and accurate understanding of their surroundings. Transformers have recently shown encouraging results in image segmentation. However, transformer-based models are challenging to strike a better balance between performance and efficiency. The computational complexity of the transformer-based models is quadratic with the number of inputs, which severely hinders their application in dense prediction tasks. In this paper, we present the semantic-aware dimension-pooling transformer (SDPT) to mitigate the conflict between accuracy and efficiency. The proposed model comprises an efficient transformer encoder for generating hierarchical features and a semantic-balanced decoder for predicting semantic masks. In the encoder, a dimension-pooling mechanism is used in the multi-head self-attention (MHSA) to reduce the computational cost, and a parallel depth-wise convolution is used to capture local semantics. Simultaneously, we further apply this dimension-pooling attention (DPA) to the decoder as a refinement module to integrate multi-level features. With such a simple yet powerful encoder-decoder framework, we empirically demonstrate that the proposed SDPT achieves excellent performance and efficiency on various popular benchmarks, including ADE20K, Cityscapes, and COCO-Stuff. For example, our SDPT achieves 48.6% mIOU on the ADE20K dataset, which outperforms the current methods with fewer computational costs. The codes can be found at https://github.com/HuCaoFighting/SDPT.


    Zugriff

    Download


    Exportieren, teilen und zitieren



    Titel :

    SDPT: Semantic-Aware Dimension-Pooling Transformer for Image Segmentation


    Beteiligte:
    Cao, Hu (Autor:in) / Chen, Guang (Autor:in) / Zhao, Hengshuang (Autor:in) / Jiang, Dongsheng (Autor:in) / Zhang, Xiaopeng (Autor:in) / Tian, Qi (Autor:in) / Knoll, Alois (Autor:in)

    Erschienen in:

    Erscheinungsdatum :

    01.11.2024


    Format / Umfang :

    2542242 byte




    Medientyp :

    Aufsatz (Zeitschrift)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch






    Situation-Aware Model Refinement for Semantic Image Segmentation

    Habermayr, Lukas / Hofbauer, Markus / Zacchi, Joao-Vitor et al. | IEEE | 2021


    Hyperbolic Uncertainty Aware Semantic Segmentation

    Chen, Bike / Peng, Wei / Cao, Xiaofeng et al. | IEEE | 2024


    TransRVNet: LiDAR Semantic Segmentation With Transformer

    Cheng, Hui-Xian / Han, Xian-Feng / Xiao, Guo-Qiang | IEEE | 2023