The recognition and utilization of complementary information arising from modality-intrinsic properties play crucial roles in multimodal 3D detection. However, most of the current approaches for fusion-based 3D detection follow symmetrical fusion paradigms and adopt early fusion, middle fusion as well as late fusion styles, which ignore the unequal status of data with different modalities. In this paper, according to the timing of fusion, we adopt an asymmetric cascade fusion network to exploit both the structural information from point clouds and the complementary semantic information from images. A multi-stage cascade design of 3D object detection is proposed to iteratively refine predictions and several late image features (comprised of detection clues, segmentation clues, and deep features from encoders) are incorporated into different stages of the LiDAR branch to maintain the integrity of image features and enable deep multimodal interactions. Besides, to mitigate the effects of the down-sampling of voxelized features and possible mismatching of multimodal data, we propose proxy-based cross-modality sampling to utilize the high-density point clouds coordinates and develop an image degeneration process to simulate the noise in cross-modality matching for robust training. Extensive experiments are conducted on KITTI and Waymo Open Dataset, which validate the effectiveness of the proposed method.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    ACF-Net: Asymmetric Cascade Fusion for 3D Detection With LiDAR Point Clouds and Images


    Contributors:
    Tian, Yonglin (author) / Zhang, Xianjing (author) / Wang, Xiao (author) / Xu, Jintao (author) / Wang, Jiangong (author) / Ai, Rui (author) / Gu, Weihao (author) / Ding, Weiping (author)

    Published in:

    Publication date :

    2024-02-01


    Size :

    2339531 byte




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English



    Fusion of Lidar and Stereo Point Clouds using Bayesian Networks

    Schimpf, Felix | German Aerospace Center (DLR) | 2018

    Free access

    Top-down object detection from LiDAR point clouds

    SMOLYANSKIY NIKOLAI / OLDJA RYAN / CHEN KE et al. | European Patent Office | 2024

    Free access

    Detection of Cars in Mobile Lidar Point Clouds

    Li, Guorui / Fang, Xinwei / Khoshelham, Kourosh et al. | IEEE | 2018


    TOP-DOWN OBJECT DETECTION FROM LIDAR POINT CLOUDS

    SMOLYANSKIY NIKOLAI / OLDJA RYAN / CHEN KE et al. | European Patent Office | 2024

    Free access

    TOP-DOWN OBJECT DETECTION FROM LIDAR POINT CLOUDS

    SMOLYANSKIY NIKOLAI / OLDJA RYAN / CHEN KE et al. | European Patent Office | 2021

    Free access