Multi-modal fused images can provide reliable perceptual information for intelligent vehicles in various weather and lighting conditions. However, most existing fusion algorithms neglect the information interactions among different modalities, leading to a loss of essential information in transportation systems characterized by strong information correlations. To enhance the quality of multi-modal semantic information fusion perception in intelligent vehicles, we propose the Cross Rubik Cube Attention Fusion Network (RCAFusion). Inspired by the shape and recovery process of a Rubik’s Cube, RCAFusion establishes an information interaction pathway among different modalities, and it achieves a more comprehensive information crossover through the simultaneous spatial attention, channel attention, and self-attention mechanisms, which enhance the feature extraction effect in the fusion architecture. Experimental results demonstrate that RCAFusion outperforms mainstream fusion algorithms in several metrics and obtains the highest score in the objective fused image metric Qabf. Moreover, the fused images output by RCAFusion have good results in the image object detection task and can achieve 95.4% mAP using the yolov8m model in the MSRS open source datasets. Pre-trained model and code are available at https://github.com/vehicle-AngLi/RCAFusion.
RCAFusion: Cross Rubik Cube Attention Network for Multi-modal Image Fusion of Intelligent Vehicles
2024 IEEE Intelligent Vehicles Symposium (IV) ; 2848-2854
02.06.2024
4709769 byte
Aufsatz (Konferenz)
Elektronische Ressource
Englisch
RUBIK - Anschlusssicherung und Echtzeitinformation im regionalen Busverkehr
Tema Archiv | 2013
|RUBIK — Anschlusssicherung und Echtzeitinformation im regionalen Busverkehr
Online Contents | 2013
|British Library Conference Proceedings | 2021
|