Vehicle re-identification (Re-ID) is a critical task in intelligent transportation, aiming to match vehicle images of the same identity captured by non-overlapping cameras. However, it is difficult to achieve satisfactory results based on RGB images alone in darkness. Therefore, it is of great importance to consider multi-modality vehicle re-identification. Currently, the proposed works deal with different modality features through direct summation and fusion based on heat map, which however ignores the relationship between them. Meanwhile, there is a huge gap between the different modalities, which needs to be reduced. In this paper, to solve the above two problems, we propose a Graph-based Progressive Fusion Network (GPFNet) using a graph convolutional network to adaptively fuse multi-modality features in an end-to-end learning framework. GPFNet consists of a CNN feature extraction module (FEM), a GCN feature fusion module (FFM), and a loss function module (LFM). Firstly, in FEM, we employ a multi-stream network architecture to extract single-modality features and common-modality features and employ a random modality substitution module to extract mixed-modality features. Secondly, in FFM, we design an efficient graph structure to associate the features of different modalities and adopt a progressive two-stage strategy to fuse them. Finally, in LFM, we use GCN-aware multi-modality loss to constrain the features. For reducing modality differences and contributing better initial mixed-modality features to FFM, we propose random modality substitution as a data enhancement method for multi-modality datasets. Extensive experiments on multi-modality vehicle Re-ID datasets RGBN300 and RGBNT100 show that our model achieves state-of-the-art performance.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    Graph-Based Progressive Fusion Network for Multi-Modality Vehicle Re-Identification


    Beteiligte:
    He, Qiaolin (Autor:in) / Lu, Zefeng (Autor:in) / Wang, Zihan (Autor:in) / Hu, Haifeng (Autor:in)

    Erschienen in:

    Erscheinungsdatum :

    01.11.2023


    Format / Umfang :

    2420611 byte




    Medientyp :

    Aufsatz (Zeitschrift)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch



    Machine Learning Multi-Modality Fusion Approaches Outperform Single-Modality & Traditional Approaches

    Garagic, Denis / Pelgrift, Daniel / Peskoe, Jacob et al. | IEEE | 2021



    Fusion of multi-modality volumetric medical imagery

    Aguilar, M. / New, J.R. | IEEE | 2002



    Fusion of Multi-Modality Volumetric Medical Imagery

    Aguilar, M. / International Society of Information Fusion / Institute of Electrical and Electronics Engineers | British Library Conference Proceedings | 2002