LIDAR and RGB cameras are commonly used sensors in autonomous vehicles. However, both of them have limitations: LIDAR provides accurate depth but is sparse in vertical and horizontal resolution; RGB images provide dense texture but lack depth information. In this paper, we fuse LIDAR and RGB images by a deep neural network, which completes a denser pixel-wise depth map. The proposed architecture reconstructs the pixel-wise depth map, taking advantage of both the dense color features and sparse 3D spatial features. We applied the early fusion technique and fine-tuned the ResNet model as the encoder. The designed Residual Up-Projection block recovers the spatial resolution of the feature map and captures context within the depth map. We introduced a depth feature tensor which propagates context information from encoder blocks to decoder blocks. Our proposed method is evaluated on the large-scale indoor NYUdepthV2 and KITTI odometry datasets and outperforms the state-of-the-art single RGB image and depth fusion method. The proposed method is also evaluated on a reduced-resolution KITTI dataset which synthesizes the planar LIDAR and RGB image fusion.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    LIDAR and Monocular Camera Fusion: On-road Depth Completion for Autonomous Driving


    Beteiligte:
    Fu, Chen (Autor:in) / Mertz, Christoph (Autor:in) / Dolan, John M. (Autor:in)


    Erscheinungsdatum :

    01.10.2019


    Format / Umfang :

    2831712 byte




    Medientyp :

    Aufsatz (Konferenz)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch




    Monocular Fisheye Camera Depth Estimation Using Sparse LiDAR Supervision

    Kumar, Varun Ravi / Milz, Stefan / Witt, Christian et al. | IEEE | 2018


    Self-Supervised Depth Completion From Direct Visual-LiDAR Odometry in Autonomous Driving

    Song, Zhenbo / Lu, Jianfeng / Yao, Yazhou et al. | IEEE | 2022


    A variational approach for estimation of monocular depth and camera motion in autonomous driving

    Hu, Huijuan / Hu, Chuan / Zhang, Xuetao | SAGE Publications | 2022

    Freier Zugriff

    DIRTY MONOCULAR CAMERA DETECTION METHODS FOR AUTONOMOUS DRIVING VEHICLE

    ZHU FAN | Europäisches Patentamt | 2021

    Freier Zugriff