LIDAR and RGB cameras are commonly used sensors in autonomous vehicles. However, both of them have limitations: LIDAR provides accurate depth but is sparse in vertical and horizontal resolution; RGB images provide dense texture but lack depth information. In this paper, we fuse LIDAR and RGB images by a deep neural network, which completes a denser pixel-wise depth map. The proposed architecture reconstructs the pixel-wise depth map, taking advantage of both the dense color features and sparse 3D spatial features. We applied the early fusion technique and fine-tuned the ResNet model as the encoder. The designed Residual Up-Projection block recovers the spatial resolution of the feature map and captures context within the depth map. We introduced a depth feature tensor which propagates context information from encoder blocks to decoder blocks. Our proposed method is evaluated on the large-scale indoor NYUdepthV2 and KITTI odometry datasets and outperforms the state-of-the-art single RGB image and depth fusion method. The proposed method is also evaluated on a reduced-resolution KITTI dataset which synthesizes the planar LIDAR and RGB image fusion.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    LIDAR and Monocular Camera Fusion: On-road Depth Completion for Autonomous Driving


    Contributors:


    Publication date :

    2019-10-01


    Size :

    2831712 byte




    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English




    Self-Supervised Depth Completion From Direct Visual-LiDAR Odometry in Autonomous Driving

    Song, Zhenbo / Lu, Jianfeng / Yao, Yazhou et al. | IEEE | 2022


    Monocular Fisheye Camera Depth Estimation Using Sparse LiDAR Supervision

    Kumar, Varun Ravi / Milz, Stefan / Witt, Christian et al. | IEEE | 2018


    A variational approach for estimation of monocular depth and camera motion in autonomous driving

    Hu, Huijuan / Hu, Chuan / Zhang, Xuetao | SAGE Publications | 2022

    Free access

    DIRTY MONOCULAR CAMERA DETECTION METHODS FOR AUTONOMOUS DRIVING VEHICLE

    ZHU FAN | European Patent Office | 2021

    Free access