As a key task in autonomous driving, 3D object detection based on LiDAR-camera fusion is expected to achieve more robust results by the complementarity of the two sensors. However, LiDAR-camera fusion is non-trivial. An existing problem for this type of detector is that the scale and receptive field of LiDAR point features and image features are not matched, leading to information deficiency or redundancy in fusion. This paper proposes a Point-based Pyramid Attention Fusion (PPAF) module for LiDAR-camera fusion to solve the problem. The PPAF module learns corresponding image features of LiDAR points with a matched scale based on the image feature pyramid and attention mechanism for a better effect of fusion. Furthermore, based on the PPAF module, a new LiDAR-camera fusion-based 3D object detector named FS-Net is proposed, a two-stage detector with LiDAR voxel-based RPN and refinement network based on enriched LiDAR-camera features. Experiments on two public datasets demonstrate the effectiveness of our approach.
FS-Net: LiDAR-Camera Fusion With Matched Scale for 3D Object Detection in Autonomous Driving
IEEE Transactions on Intelligent Transportation Systems ; 24 , 11 ; 12154-12165
2023-11-01
19610477 byte
Article (Journal)
Electronic Resource
English
ONLINE CAMERA LIDAR FUSION AND OBJECT DETECTION ON HYBRID DATA FOR AUTONOMOUS DRIVING
British Library Conference Proceedings | 2018
|LiDAR and Camera-Based Convolutional Neural Network Detection for Autonomous Driving
British Library Conference Proceedings | 2020
|Panoptic Based Camera and Lidar Fusion for Distance Estimation in Autonomous Driving Vehicles
British Library Conference Proceedings | 2022
|