Research in the field of Continual Semantic Segmentation is mainly investigating novel learning algorithms to overcome catastrophic forgetting of neural networks. Most recent publications have focused on improving learning algorithms without distinguishing effects caused by the choice of neural architecture. Therefore, we study how the choice of neural network architecture affects catastrophic forgetting in class- and domain-incremental semantic segmentation. Specifically, we compare the well-researched CNNs to recently proposed Transformers and Hybrid architectures, as well as the impact of the choice of novel normalization layers and different decoder heads. We find that traditional CNNs like ResNet have high plasticity but low stability, while transformer architectures are much more stable. When the inductive biases of CNN architectures are combined with transformers in hybrid architectures, it leads to higher plasticity and stability of the model. The stability of these models can be explained by their ability to learn general features which are robust against distribution shifts. Experiments with different normalization layers show that Continual Normalization achieves the best trade-off in terms of adaptability and stability of the model, especially in domain-incremental learning. Our experiments suggest that the right choice of architecture can significantly reduce forgetting even with naive fine-tuning and confirm that for real-world applications, the architecture is an important factor in designing a continual learning model.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Effects of Architectures on Continual Semantic Segmentation


    Contributors:


    Publication date :

    2023-06-04


    Size :

    1229819 byte





    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English



    Continual BatchNorm Adaptation (CBNA) for Semantic Segmentation

    Klingner, Marvin / Ayache, Mouadh / Fingscheidt, Tim | IEEE | 2022

    Free access

    Continual Learning for Class- and Domain-Incremental Semantic Segmentation

    Kalb, Tobias / Roschani, Masoud / Ruf, Miriam et al. | IEEE | 2021


    Improving Replay-Based Continual Semantic Segmentation with Smart Data Selection

    Kalb, Tobias / Mauthe, Bjorn / Beyerer, Jurgen | IEEE | 2022


    Continual Unsupervised Domain Adaptation for Semantic Segmentation by Online Frequency Domain Style Transfer

    Termohlen, Jan-Aike / Klingner, Marvin / Brettin, Leon J. et al. | IEEE | 2021


    CLSD: Continual Learning for Lane Line Segmentation Across Domains

    Wu, Yan / Liao, Yujun / Jiang, Wei et al. | IEEE | 2022