Using images captured by drone cameras and comparing them with known Google satellite maps to obtain the current location of the drone is an important way of UAV navigation in GPS-denied environments. But, due to inherent modality differences and significant geometric deformations, cross-modal image registration is challenging. This paper proposes a CNN-Transformer hybrid network model for feature detection and feature matching. ResNet50 is used as the backbone network for feature extraction. An improved feature fusion module is used to fuse feature maps from different levels, and then a Transformer encoder–decoder structure is used for feature matching to obtain preliminary correspondences. Finally, a geometric outlier removal method (GSM) is used to eliminate mismatched points based on the geometric similarity of inliers, resulting in more robust correspondences. Qualitative and quantitative experiments were conducted on multimodal image datasets captured by UAVs; the correct matching rate was improved by 52%, 21%, and 15%, respectively, and the error was reduced by 36% compared to the 3MRS algorithm. A total of 56 experiments were conducted in actual scenarios, with a localization success rate of 91.1%, and the RMSE of UAV positioning was 4.6 m.


    Access

    Download


    Export, share and cite



    Title :

    A Multimodal Image Registration Method for UAV Visual Navigation Based on Feature Fusion and Transformers


    Contributors:
    Ruofei He (author) / Shuangxing Long (author) / Wei Sun (author) / Hongjuan Liu (author)


    Publication date :

    2024




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    Unknown




    Incorporating global information in feature-based multimodal image registration

    Li, Y. / Stevenson, R. | British Library Online Contents | 2014



    Multi-type feature fusion visual navigation for asteroid landing

    XIU, Wenbo / ZHU, Shengying / LIU, Yanjie | Elsevier | 2025

    Free access

    3D ultrasound registration-based visual servoing for neurosurgical navigation.

    Zettinig, Oliver / Frisch, Benjamin / Virga, Salvatore et al. | BASE | 2018

    Free access

    Multimodal Functional and Morphological Nonrigid Image Registration

    Bernardes, R. / Baptista, P. / Cunha-Vaz, J. et al. | British Library Conference Proceedings | 2005