Obstacle perception based on radar sensor has drawn wide attentions in autonomous driving due to robust performance and low cost. It is significant to utilize fusion, e.g., camera information, to further enhance the radar perception ability. Although much progress has been made, we still observe two problems: First, the spatial alignment among multi-modal data is intractable when involving multiple radar and camera sensors. Second, most existing works are based on object-level fusion, which inevitably has information loss leading to a performance degradation. To this end, we propose a feature-level fusion detection framework based on multiple radars and cameras, termed as the RVDet. We first establish an occupancy grid map by using 4 corner radars and extract radar features in the bird's eye view(BEV). Meantime, the image features of 4 fish-eye cameras are obtained using a pretraining vision detection model. Then, an adaptive projection network is employed to transform all the 4 image features to the BEV domain and integrate them to a dense spatial feature map aligned with the radar feature. Last, the carefully aligned multi-modal feature maps are jointly sent to a deep fusion network to predict final fused detection results. Experiments show that both object detection and positioning performance achieve significant gains by the proposed method in a custom dataset.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    RVDet: Feature-level Fusion of Radar and Camera for Object Detection


    Beteiligte:
    Zhang, Jingwei (Autor:in) / Zhang, Ming (Autor:in) / Fang, Zicheng (Autor:in) / Wang, Yulong (Autor:in) / Zhao, Xian (Autor:in) / Pu, Shiliang (Autor:in)


    Erscheinungsdatum :

    2021-09-19


    Format / Umfang :

    2620553 byte




    Medientyp :

    Aufsatz (Konferenz)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch



    Deep Learning-based Radar, Camera, and Lidar Fusion for Object Detection

    Nobis, Felix Otto Geronimo | TIBKAT | 2022

    Freier Zugriff

    OBJECT DETECTION METHOD BASED ON RADAR-CAMERA FUSION AND ELECTRONIC APPARATUS

    CHUNG YOUN KYU / PARK YONG JEONG / CUI CHENGLIN et al. | Europäisches Patentamt | 2022

    Freier Zugriff

    3D Multi-Object Tracking Based on Radar-Camera Fusion

    Lin, Zihao / Hu, Jianming | IEEE | 2022


    Automated vehicle object detection system with camera image and radar data fusion

    CHEN SUSAN / SCHWARTZ DAVID A / MANGALGIRI ANURADHA et al. | Europäisches Patentamt | 2018

    Freier Zugriff

    A Concise Camera-Radar Fusion Framework for Object Detection and Data Association

    Zhao, Jian / Li, Linhui / Lyu, Nana et al. | SAE Technical Papers | 2022