In the context of autonomous driving environment perception, multi-modal fusion plays a pivotal role in enhancing robustness, completeness, and accuracy, thereby extending the performance boundary of the perception system. However, directly applying LiDAR-related algorithms to radar and camera fusion leads to significant challenges, such as radar sparsity, absence of height information, and noise, resulting in substantial performance loss. To address these issues, our proposed method, SparseFusion3D, utilizes a dual-branch feature-level fusion network that fully models sensor interactions, effectively mitigating the adverse effects of radar sparsity and noise on modality association. Additionally, to enhance modal correlations and accuracy while alleviating radar point cloud sparsity and measurement ambiguity, we introduce MSPCP, which compensates for point cloud offset. Moreover, we integrate Radar Painter to leverage image information and further enhance MSPCP. SparseFusion3D exhibits competitive performance compared to previous radar-camera fusion models, achieving approximately 1.5x inference speedup with similar performance to dense query methods, while also improving by 20.1% compared to the baseline approach.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    SparseFusion3D: Sparse Sensor Fusion for 3D Object Detection by Radar and Camera in Environmental Perception


    Contributors:
    Yu, Zedong (author) / Wan, Weibing (author) / Ren, Maiyu (author) / Zheng, Xiuyuan (author) / Fang, Zhijun (author)

    Published in:

    Publication date :

    2024-01-01


    Size :

    3175105 byte




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English



    Robust and efficient multi-object detection and tracking for vehicle perception systems using radar and camera sensor fusion

    Burlet, J. / Fontana, M.D. / Institution of Engineering and Technology | British Library Conference Proceedings | 2012


    RVDet: Feature-level Fusion of Radar and Camera for Object Detection

    Zhang, Jingwei / Zhang, Ming / Fang, Zicheng et al. | IEEE | 2021


    Deep Learning-based Radar, Camera, and Lidar Fusion for Object Detection

    Nobis, Felix Otto Geronimo | TIBKAT | 2022

    Free access

    OBJECT DETECTION METHOD BASED ON RADAR-CAMERA FUSION AND ELECTRONIC APPARATUS

    CHUNG YOUN KYU / PARK YONG JEONG / CUI CHENGLIN et al. | European Patent Office | 2022

    Free access

    A Concise Camera-Radar Fusion Framework for Object Detection and Data Association

    Zhao, Jian / Li, Linhui / Lyu, Nana et al. | SAE Technical Papers | 2022