Cross-modal hashing is an effective cross-modal retrieval approach because of its low storage and high efficiency. However, most existing methods mainly utilize pre-trained networks to extract modality-specific features, while ignore the position information and lack information interaction between different modalities. To address those problems, in this paper, we propose a novel approach, named cross-modal generation and pair correlation alignment hashing (CMGCAH), which introduces transformer to exploit position information and utilizes cross-modal generative adversarial networks (GAN) to boost cross-modal information interaction. Concretely, a cross-modal interaction network based on conditional generative adversarial network and pair correlation alignment networks are proposed to generate cross-modal common representations. On the other hand, a transformer-based feature extraction network (TFEN) is designed to exploit position information, which can be propagated to text modality and enforce the common representation to be semantically consistent. Experiments are performed on widely used datasets with text-image modalities, and results show that the proposed method achieved competitive performance compared with many existing methods.
Cross-Modal Generation and Pair Correlation Alignment Hashing
IEEE Transactions on Intelligent Transportation Systems ; 24 , 3 ; 3018-3026
2023-03-01
3206677 byte
Article (Journal)
Electronic Resource
English
Label Guided Discrete Hashing for Cross-Modal Retrieval
IEEE | 2022
|Asymptotic modal analysis of dynamical systems: the effect of modal cross-correlation
Online Contents | 2004
|British Library Conference Proceedings | 2023
|British Library Conference Proceedings | 2015
|