The recognition and utilization of complementary information arising from modality-intrinsic properties play crucial roles in multimodal 3D detection. However, most of the current approaches for fusion-based 3D detection follow symmetrical fusion paradigms and adopt early fusion, middle fusion as well as late fusion styles, which ignore the unequal status of data with different modalities. In this paper, according to the timing of fusion, we adopt an asymmetric cascade fusion network to exploit both the structural information from point clouds and the complementary semantic information from images. A multi-stage cascade design of 3D object detection is proposed to iteratively refine predictions and several late image features (comprised of detection clues, segmentation clues, and deep features from encoders) are incorporated into different stages of the LiDAR branch to maintain the integrity of image features and enable deep multimodal interactions. Besides, to mitigate the effects of the down-sampling of voxelized features and possible mismatching of multimodal data, we propose proxy-based cross-modality sampling to utilize the high-density point clouds coordinates and develop an image degeneration process to simulate the noise in cross-modality matching for robust training. Extensive experiments are conducted on KITTI and Waymo Open Dataset, which validate the effectiveness of the proposed method.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    ACF-Net: Asymmetric Cascade Fusion for 3D Detection With LiDAR Point Clouds and Images


    Beteiligte:
    Tian, Yonglin (Autor:in) / Zhang, Xianjing (Autor:in) / Wang, Xiao (Autor:in) / Xu, Jintao (Autor:in) / Wang, Jiangong (Autor:in) / Ai, Rui (Autor:in) / Gu, Weihao (Autor:in) / Ding, Weiping (Autor:in)

    Erschienen in:

    Erscheinungsdatum :

    2024-02-01


    Format / Umfang :

    2339531 byte




    Medientyp :

    Aufsatz (Zeitschrift)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch



    Fusion of Lidar and Stereo Point Clouds using Bayesian Networks

    Schimpf, Felix | Deutsches Zentrum für Luft- und Raumfahrt (DLR) | 2018

    Freier Zugriff

    TOP-DOWN OBJECT DETECTION FROM LIDAR POINT CLOUDS

    SMOLYANSKIY NIKOLAI / OLDJA RYAN / CHEN KE et al. | Europäisches Patentamt | 2021

    Freier Zugriff

    Detection of Cars in Mobile Lidar Point Clouds

    Li, Guorui / Fang, Xinwei / Khoshelham, Kourosh et al. | IEEE | 2018


    3D Vehicle Detection Using Multi-Level Fusion From Point Clouds and Images

    Zhao, Kun / Ma, Lingfei / Meng, Yu et al. | IEEE | 2022


    Detection of Condensed Vehicle Gas Exhaust in LiDAR Point Clouds

    Piroli, Aldi / Dallabetta, Vinzenz / Walessa, Marc et al. | IEEE | 2022