Obstacle perception based on radar sensor has drawn wide attentions in autonomous driving due to robust performance and low cost. It is significant to utilize fusion, e.g., camera information, to further enhance the radar perception ability. Although much progress has been made, we still observe two problems: First, the spatial alignment among multi-modal data is intractable when involving multiple radar and camera sensors. Second, most existing works are based on object-level fusion, which inevitably has information loss leading to a performance degradation. To this end, we propose a feature-level fusion detection framework based on multiple radars and cameras, termed as the RVDet. We first establish an occupancy grid map by using 4 corner radars and extract radar features in the bird's eye view(BEV). Meantime, the image features of 4 fish-eye cameras are obtained using a pretraining vision detection model. Then, an adaptive projection network is employed to transform all the 4 image features to the BEV domain and integrate them to a dense spatial feature map aligned with the radar feature. Last, the carefully aligned multi-modal feature maps are jointly sent to a deep fusion network to predict final fused detection results. Experiments show that both object detection and positioning performance achieve significant gains by the proposed method in a custom dataset.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    RVDet: Feature-level Fusion of Radar and Camera for Object Detection


    Contributors:
    Zhang, Jingwei (author) / Zhang, Ming (author) / Fang, Zicheng (author) / Wang, Yulong (author) / Zhao, Xian (author) / Pu, Shiliang (author)


    Publication date :

    2021-09-19


    Size :

    2620553 byte




    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English



    Deep Learning-based Radar, Camera, and Lidar Fusion for Object Detection

    Nobis, Felix Otto Geronimo | TIBKAT | 2022

    Free access

    OBJECT DETECTION METHOD BASED ON RADAR-CAMERA FUSION AND ELECTRONIC APPARATUS

    CHUNG YOUN KYU / PARK YONG JEONG / CUI CHENGLIN et al. | European Patent Office | 2022

    Free access

    A Concise Camera-Radar Fusion Framework for Object Detection and Data Association

    Zhao, Jian / Li, Linhui / Lyu, Nana et al. | SAE Technical Papers | 2022


    Automated vehicle object detection system with camera image and radar data fusion

    CHEN SUSAN / SCHWARTZ DAVID A / MANGALGIRI ANURADHA et al. | European Patent Office | 2018

    Free access

    A Concise Camera-Radar Fusion Framework for Object Detection and Data Association

    He, Yingjie / Zhao, Jian / Lyu, Nana et al. | British Library Conference Proceedings | 2022