As a key task in autonomous driving, 3D object detection based on LiDAR-camera fusion is expected to achieve more robust results by the complementarity of the two sensors. However, LiDAR-camera fusion is non-trivial. An existing problem for this type of detector is that the scale and receptive field of LiDAR point features and image features are not matched, leading to information deficiency or redundancy in fusion. This paper proposes a Point-based Pyramid Attention Fusion (PPAF) module for LiDAR-camera fusion to solve the problem. The PPAF module learns corresponding image features of LiDAR points with a matched scale based on the image feature pyramid and attention mechanism for a better effect of fusion. Furthermore, based on the PPAF module, a new LiDAR-camera fusion-based 3D object detector named FS-Net is proposed, a two-stage detector with LiDAR voxel-based RPN and refinement network based on enriched LiDAR-camera features. Experiments on two public datasets demonstrate the effectiveness of our approach.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    FS-Net: LiDAR-Camera Fusion With Matched Scale for 3D Object Detection in Autonomous Driving


    Contributors:
    Zhang, Lei (author) / Li, Xu (author) / Tang, Kaichen (author) / Jiang, Yunzhe (author) / Yang, Liu (author) / Zhang, Yonggang (author) / Chen, Xianyi (author)

    Published in:

    Publication date :

    2023-11-01


    Size :

    19610477 byte




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English



    ONLINE CAMERA LIDAR FUSION AND OBJECT DETECTION ON HYBRID DATA FOR AUTONOMOUS DRIVING

    Banerjee, Koyel / Notz, Dominik / Windelen, Johannes et al. | British Library Conference Proceedings | 2018


    Online Camera LiDAR Fusion and Object Detection on Hybrid Data for Autonomous Driving

    Banerjee, Koyel / Notz, Dominik / Windelen, Johannes et al. | IEEE | 2018


    LIDAR and Monocular Camera Fusion: On-road Depth Completion for Autonomous Driving

    Fu, Chen / Mertz, Christoph / Dolan, John M. | IEEE | 2019


    LiDAR and Camera-Based Convolutional Neural Network Detection for Autonomous Driving

    Hamieh, Ismail / Myers, Ryan / Nimri, Hisham et al. | British Library Conference Proceedings | 2020


    Panoptic Based Camera and Lidar Fusion for Distance Estimation in Autonomous Driving Vehicles

    Jose, Edwin / P, Aparna M / Patil, Mrinalini et al. | British Library Conference Proceedings | 2022