LiDAR-Camera fusion can effectively provide the complementary geometric and appearance information for 3D object detection task of autonomous driving system. However, current dominant methods designed for urban scenes often fuse multi-modal information into a Bird's-Eye-View (BEV) feature representation and have a relatively short perception range, which limits their application in surface mine. To achieve long-range perception in surface mine, we utilize a solid-state LiDAR (i.e. Livox Horizon) that has a large perception range (over 200 meters). Accordingly, we propose PVFusion, a novel 3D object detection method that only fuses LiDAR and camera information in Perspective View (PV). Specifically, we first generate the multi-channel depth map via point cloud projection and Depth Map Densification (DMD). Then, we extract visual-depth features and local geometry features from depth-enhanced camera image and point cloud respectively. To better fuse multi-modal information, we propose an Attentional Feature Fusion (AFF) module to dynamically fuse visual-depth features and local geometry features into fused PV features. Moreover, we design a three-branch detection head that leverages a Depth Decoupling Strategy (DDS) to improve the depth estimation for distant objects. We conduct extensive experiments on our surface mine dataset and verify the effectiveness of the proposed PVFusion.
LiDAR-Camera Fusion in Perspective View for 3D Object Detection in Surface Mine
IEEE Transactions on Intelligent Vehicles ; 9 , 2 ; 3721-3730
2024-02-01
2244250 byte
Aufsatz (Zeitschrift)
Elektronische Ressource
Englisch
Deep Learning-based Radar, Camera, and Lidar Fusion for Object Detection
TIBKAT | 2022
|Spatial aware object detection by LIDAR and camera fusion based super-resolution
Europäisches Patentamt | 2024
|SPATIAL AWARE OBJECT DETECTION BY LIDAR AND CAMERA FUSION BASED SUPER-RESOLUTION
Europäisches Patentamt | 2021
|RADAR LIDAR OBJECT DETECTION USING RADAR AND LIDAR FUSION
Europäisches Patentamt | 2023
|