LiDAR-Camera fusion can effectively provide the complementary geometric and appearance information for 3D object detection task of autonomous driving system. However, current dominant methods designed for urban scenes often fuse multi-modal information into a Bird's-Eye-View (BEV) feature representation and have a relatively short perception range, which limits their application in surface mine. To achieve long-range perception in surface mine, we utilize a solid-state LiDAR (i.e. Livox Horizon) that has a large perception range (over 200 meters). Accordingly, we propose PVFusion, a novel 3D object detection method that only fuses LiDAR and camera information in Perspective View (PV). Specifically, we first generate the multi-channel depth map via point cloud projection and Depth Map Densification (DMD). Then, we extract visual-depth features and local geometry features from depth-enhanced camera image and point cloud respectively. To better fuse multi-modal information, we propose an Attentional Feature Fusion (AFF) module to dynamically fuse visual-depth features and local geometry features into fused PV features. Moreover, we design a three-branch detection head that leverages a Depth Decoupling Strategy (DDS) to improve the depth estimation for distant objects. We conduct extensive experiments on our surface mine dataset and verify the effectiveness of the proposed PVFusion.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    LiDAR-Camera Fusion in Perspective View for 3D Object Detection in Surface Mine


    Contributors:
    Ai, Yunfeng (author) / Yang, Xue (author) / Song, Ruiqi (author) / Cui, Chenglin (author) / Li, Xinqing (author) / Cheng, Qi (author) / Tian, Bin (author) / Chen, Long (author)

    Published in:

    Publication date :

    2024-02-01


    Size :

    2244250 byte




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English



    Deep Learning-based Radar, Camera, and Lidar Fusion for Object Detection

    Nobis, Felix Otto Geronimo | TIBKAT | 2022

    Free access

    Spatial aware object detection by LIDAR and camera fusion based super-resolution

    KABAI ROBERT-ZSOLT | European Patent Office | 2024

    Free access

    SPATIAL AWARE OBJECT DETECTION BY LIDAR AND CAMERA FUSION BASED SUPER-RESOLUTION

    KABAI ROBERT-ZSOLT | European Patent Office | 2021

    Free access


    ONLINE CAMERA LIDAR FUSION AND OBJECT DETECTION ON HYBRID DATA FOR AUTONOMOUS DRIVING

    Banerjee, Koyel / Notz, Dominik / Windelen, Johannes et al. | British Library Conference Proceedings | 2018