In this paper, we introduce a new deep learning architecture for camera and Lidar sensor fusion. The proposed scheme performs 2D object detection using the RGB camera image and the depth, height, and intensity images generated by projecting the 3D Lidar point cloud into camera image plane. The proposed object detector consists of two convolutional neural networks (CNNs) that process the RGB and Lidar images separately as well as the fusion network that combines the feature maps produced at the intermediate layers of the CNNs. We aim to develop a robust object detector that maintains good object detection accuracy even when the quality of the sensor signals is degraded for object detection. Towards this end, we devise the gated fusion unit (GFU) that adjusts the contribution of the feature maps generated by two CNN structures via gating mechanism. Using the GFU, the proposed object detector can fuse the high level feature maps drawn from two modalities with appropriate weights to achieve robust performance. Experiments conducted on the challenging KITTI benchmark show that the proposed camera and Lidar fusion network outperforms the conventional sensor fusion methods even when either of the camera and Lidar sensor signals is corrupted by missing data, occlusion, noise, and illumination change.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    Robust Camera Lidar Sensor Fusion Via Deep Gated Information Fusion Network


    Beteiligte:
    Kim, Jaekyum (Autor:in) / Choi, Jaehyung (Autor:in) / Kim, Yechol (Autor:in) / Koh, Junho (Autor:in) / Chung, Chung Choo (Autor:in) / Choi, Jun Won (Autor:in)


    Erscheinungsdatum :

    2018-06-01


    Format / Umfang :

    3840012 byte




    Medientyp :

    Aufsatz (Konferenz)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch



    Deep fusion of camera and LIDAR

    Schneider, Nick | TIBKAT | 2023

    Freier Zugriff

    Camera-LIDAR Integration: Probabilistic Sensor Fusion for Semantic Mapping

    Berrio, Julie Stephany / Shan, Mao / Worrall, Stewart et al. | IEEE | 2022


    LIDAR-Camera Fusion Where LIDAR and Camera Validly See Different Things

    BLAU CHAIM / SPRINGER OFER / ROSENBLUM KEVIN et al. | Europäisches Patentamt | 2022

    Freier Zugriff

    Deep Learning-based Radar, Camera, and Lidar Fusion for Object Detection

    Nobis, Felix Otto Geronimo | TIBKAT | 2022

    Freier Zugriff

    Multi-Classifier Based LIDAR and Camera Fusion

    Hwang, Jae Pil / Cho, Seung Eun / Ryu, Kyung Jin et al. | IEEE | 2007