Multispectral object detection has achieved remarkable results due to its ability to fuse information from visible and thermal modalities in recent years. However, the existing visible-thermal datasets are constructed based on manually aligned image pairs, which cannot fully represent the challenges of real-world scenarios where image pairs are often misaligned. Existing methods for visible-thermal object detection are based on aligned data and are limited by the accuracy of registration. To address the above issues, we propose a dataset, namely DVTOD, which is a misaligned visible-thermal object detection dataset captured by drones. DVTOD includes 16 challenging attributes and 54 capture scenes. Furthermore, we introduce a cross-modal alignment detector (CMA-Det) for misaligned visible-thermal object detection. Firstly, we design an alignment network to estimate the visible-to-thermal deformation field, which is used to correct for misalignment of the corresponding visible and thermal features. Secondly, we propose a strategy called Object Search Rectification (OSR) to improve the robustness of feature alignment. To better remove the interference of complex backgrounds, a bi-directional feature correction fusion module (BFCFM) is designed to calibrate bimodal features by exploiting the correlation of channel and spatial information between two modalities. CMA-Det outperforms existing methods on the DVTOD dataset and two other visible-thermal object detection datasets.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Misaligned Visible-Thermal Object Detection: A Drone-Based Benchmark and Baseline


    Contributors:
    Song, Kechen (author) / Xue, Xiaotong (author) / Wen, Hongwei (author) / Ji, Yingying (author) / Yan, Yunhui (author) / Meng, Qinggang (author)

    Published in:

    Publication date :

    2024-11-01


    Size :

    7923417 byte




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English



    Drone-Based Visible–Thermal Object Detection with Transformers and Prompt Tuning

    Rui Chen / Dongdong Li / Zhinan Gao et al. | DOAJ | 2024

    Free access

    Human running detection: Benchmark and baseline

    Lao, Shihong / Wang, Dong / li, Fu et al. | British Library Online Contents | 2016


    Human running detection: Benchmark and baseline

    Lao, Shihong / Wang, Dong / li, Fu et al. | British Library Online Contents | 2016


    Human running detection: Benchmark and baseline

    Lao, Shihong / Wang, Dong / li, Fu et al. | British Library Online Contents | 2016


    Drone patrol using thermal imaging for object detection

    Juang, Jih-Gau / Tu, Guan-Ting / Liao, Yu-Hsien et al. | SPIE | 2020