RGB-Thermal semantic segmentation is crucial for applications like robotic inception, autonomous driving, and video surveillance. However, small and imbalanced datasets in this field create training bias. To address this, we propose a new data augmentation scheme. We mix up the cross-modal information to smooth the decision boundaries from class to class, modal to modal using CutMix and copy-paste. Using the two method, RGB data and Thermal data are exchanged. Applying the RandAugment method to augment RGB and Thermal images, we select 2 transformations from a pool of 15, including a new transformation tailored for RGB-Thermal datasets. This innovative transformation mimics modal failure by fading a randomly selected region in either RGB or thermal images. Our experiments confirm the effectiveness of the proposed augmentation scheme, leading to a observable performance boost. Specifically, we observe improvements of 4.7% in mAcc and 6.1% in mIoU compared to the baseline.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Cross-modal Augmentation: A Data Augmentation Scheme for RGB-Thermal Semantic Segmentation


    Contributors:
    Lai, Wenjie (author) / Hu, Xiao (author) / Liu, Ziji (author) / Jiang, Yadong (author)


    Publication date :

    2023-10-11


    Size :

    4475282 byte




    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English



    Augmentation-based Domain Generalization for Semantic Segmentation

    Schwonberg, Manuel / Bouazati, Fadoua El / Schmidt, Nico M. et al. | IEEE | 2023


    Leveraging Smooth Deformation Augmentation for LiDAR Point Cloud Semantic Segmentation

    Qiu, Shoumeng / Chen, Jie / Lai, Chenghang et al. | IEEE | 2024

    Free access

    VEHICLE DATA AUGMENTATION

    SHOLINGAR GAUTHAM / SUNDAR SOWNDARYA | European Patent Office | 2021

    Free access

    Implicit Gradient-Modulated Semantic Data Augmentation for Deep Crack Recognition

    Chen, Zhuangzhuang / Lu, Ronghao / Chen, Jie et al. | IEEE | 2024


    VEHICLE DATA AUGMENTATION

    SHOLINGAR GAUTHAM / SUNDAR SUSHMITHA | European Patent Office | 2021

    Free access