Multi-modal fused images can provide reliable perceptual information for intelligent vehicles in various weather and lighting conditions. However, most existing fusion algorithms neglect the information interactions among different modalities, leading to a loss of essential information in transportation systems characterized by strong information correlations. To enhance the quality of multi-modal semantic information fusion perception in intelligent vehicles, we propose the Cross Rubik Cube Attention Fusion Network (RCAFusion). Inspired by the shape and recovery process of a Rubik’s Cube, RCAFusion establishes an information interaction pathway among different modalities, and it achieves a more comprehensive information crossover through the simultaneous spatial attention, channel attention, and self-attention mechanisms, which enhance the feature extraction effect in the fusion architecture. Experimental results demonstrate that RCAFusion outperforms mainstream fusion algorithms in several metrics and obtains the highest score in the objective fused image metric Qabf. Moreover, the fused images output by RCAFusion have good results in the image object detection task and can achieve 95.4% mAP using the yolov8m model in the MSRS open source datasets. Pre-trained model and code are available at https://github.com/vehicle-AngLi/RCAFusion.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    RCAFusion: Cross Rubik Cube Attention Network for Multi-modal Image Fusion of Intelligent Vehicles


    Contributors:
    Li, Ang (author) / Yin, Guodong (author) / Wang, Ziwei (author) / Liang, Jinhao (author) / Wang, Fanxun (author) / Bai, Xin (author) / Liu, Zhichao (author)


    Publication date :

    2024-06-02


    Size :

    4709769 byte





    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English



    RUBIK - Anschlusssicherung und Echtzeitinformation im regionalen Busverkehr

    Tritschler, Stefan / Windeisen, Horst / Dobeschinsky, Harry et al. | Tema Archive | 2013




    CACFNet: Cross-Modal Attention Cascaded Fusion Network for RGB-T Urban Scene Parsing

    Zhou, Wujie / Dong, Shaohua / Fang, Meixin et al. | IEEE | 2024