Multispectral pedestrian detection has achieved great success in past years, which can be used in autonomous driving for intelligent transportation system. Most existing multispectral pedestrian detection approaches are developed on the assumption that training and test data belong to an identical distribution, which does not guarantee a good generalization to cross-domain (unseen) data. In this paper, we aim to develop a generalizable multispectral pedestrian detector, which achieves a favorable performance on both intra-dataset evaluation and cross-dataset evaluation. To achieve this goal, we conduct intra-dataset and cross-dataset experiments using single-modal and multi-modal data. By deep analysis, we find that, compared to visible or multi-modal data, thermal data not only has a best cross-dataset generalization, but also generates high-quality proposals on intra-dataset and cross-dataset evaluations. Inspired by this, we propose a novel thermal-first and fusion-second network (called TFNet) for multispectral pedestrian detection. In our TFNet, we first employ a thermal-based proposal network to extract candidate pedestrian proposals. After that, we design a transformer fusion based head network to further classify/regress these proposals. Experiments are performed on three public datasets. The comprehensive results demonstrate the effectiveness of our proposed TFNet on both intra-dataset and cross-dataset evaluations. We hope that our simple design can promote the future study on generalizable multispectral pedestrian detection.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Toward Generalizable Multispectral Pedestrian Detection


    Contributors:
    Chu, Fuchen (author) / Cao, Jiale (author) / Song, Zhanjie (author) / Shao, Zhuang (author) / Pang, Yanwei (author) / Li, Xuelong (author)


    Publication date :

    2024-05-01


    Size :

    16359026 byte




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English



    Multispectral pedestrian detection based on feature complementation and enhancement

    Linzhen Nie / Meihe Lu / Zhiwei He et al. | DOAJ | 2024

    Free access

    Multispectral pedestrian detection based on feature complementation and enhancement

    Nie, Linzhen / Lu, Meihe / He, Zhiwei et al. | Wiley | 2024

    Free access


    An eye toward multispectral imaging

    Wolpert, H.D. | British Library Online Contents | 2011


    Generalizable Journey Mode Detection Using Unsupervised Representation Learning

    Bandyopadhyay, Soma / Datta, Anish / Ramakrishnan, Ramesh Kumar et al. | IEEE | 2024