Multispectral pedestrian detection with visible light and infrared images is robust to changes in lighting conditions and therefore is of great importance to numerous applications that require all‐day environmental perception. This paper proposes a novel method named FCE‐RCNN, which integrates saliency detection as a sub‐task and utilizes global information for enhanced feature representation. The approach enhances thermal inputs by incorporating gradients at the raw‐data level before feature extraction. Utilizing a dual‐stream backbone, a global semantic information extraction module is introduced that combines pooling with horizontal–vertical attention mechanisms, capturing high‐quality global semantic information for lower‐level feature enrichment and guidance. Additionally, the pedestrian locality enhancement module is designed to enhance spatial locality information of pedestrians through saliency detection. Furthermore, to alleviate the challenges posed by positional shifts between cross‐spectral features, deformable convolution is innovatively employed. Experimental results on the KAIST dataset demonstrate that FCE‐RCNN significantly improves nighttime detection, achieving a log‐average miss rate of 6.92%, outperforming the new method ICAFusion by 0.93%. These results underscore the effectiveness of FCE‐RCNN, and the method also maintains competitive inference speed, making it suitable for real‐time applications.


    Access

    Download


    Export, share and cite



    Title :

    Multispectral pedestrian detection based on feature complementation and enhancement


    Contributors:
    Nie, Linzhen (author) / Lu, Meihe (author) / He, Zhiwei (author) / Hu, Jiachen (author) / Yin, Zhishuai (author)

    Published in:

    Publication date :

    2024-11-01


    Size :

    12 pages




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English




    Multispectral pedestrian detection based on feature complementation and enhancement

    Linzhen Nie / Meihe Lu / Zhiwei He et al. | DOAJ | 2024

    Free access

    Toward Generalizable Multispectral Pedestrian Detection

    Chu, Fuchen / Cao, Jiale / Song, Zhanjie et al. | IEEE | 2024


    Multi-channel Feature for Pedestrian Detection

    He, Zhixiang / Xu, Meihua / Guo, Aiying | Springer Verlag | 2017


    A Multi-Feature Fusion Based Pedestrian Detection Method

    Dong, Enzeng / Jing, Cunlei / Zhang, Zufeng | British Library Conference Proceedings | 2020


    740-014: Pedestrian Detection using HOGVA Feature and Local Pedestrian Classifier

    Kasai, H. / Onoguchi, K. / International Association of Science and Technology for Development | British Library Conference Proceedings | 2011