Multispectral pedestrian detection has achieved great success in past years, which can be used in autonomous driving for intelligent transportation system. Most existing multispectral pedestrian detection approaches are developed on the assumption that training and test data belong to an identical distribution, which does not guarantee a good generalization to cross-domain (unseen) data. In this paper, we aim to develop a generalizable multispectral pedestrian detector, which achieves a favorable performance on both intra-dataset evaluation and cross-dataset evaluation. To achieve this goal, we conduct intra-dataset and cross-dataset experiments using single-modal and multi-modal data. By deep analysis, we find that, compared to visible or multi-modal data, thermal data not only has a best cross-dataset generalization, but also generates high-quality proposals on intra-dataset and cross-dataset evaluations. Inspired by this, we propose a novel thermal-first and fusion-second network (called TFNet) for multispectral pedestrian detection. In our TFNet, we first employ a thermal-based proposal network to extract candidate pedestrian proposals. After that, we design a transformer fusion based head network to further classify/regress these proposals. Experiments are performed on three public datasets. The comprehensive results demonstrate the effectiveness of our proposed TFNet on both intra-dataset and cross-dataset evaluations. We hope that our simple design can promote the future study on generalizable multispectral pedestrian detection.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    Toward Generalizable Multispectral Pedestrian Detection


    Beteiligte:
    Chu, Fuchen (Autor:in) / Cao, Jiale (Autor:in) / Song, Zhanjie (Autor:in) / Shao, Zhuang (Autor:in) / Pang, Yanwei (Autor:in) / Li, Xuelong (Autor:in)


    Erscheinungsdatum :

    01.05.2024


    Format / Umfang :

    16359026 byte




    Medientyp :

    Aufsatz (Zeitschrift)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch



    Multispectral pedestrian detection based on feature complementation and enhancement

    Linzhen Nie / Meihe Lu / Zhiwei He et al. | DOAJ | 2024

    Freier Zugriff

    Multispectral pedestrian detection based on feature complementation and enhancement

    Nie, Linzhen / Lu, Meihe / He, Zhiwei et al. | Wiley | 2024

    Freier Zugriff


    An eye toward multispectral imaging

    Wolpert, H.D. | British Library Online Contents | 2011


    Generalizable Journey Mode Detection Using Unsupervised Representation Learning

    Bandyopadhyay, Soma / Datta, Anish / Ramakrishnan, Ramesh Kumar et al. | IEEE | 2024