For applications in robotics and autonomous vehicles, fusing 3D LiDAR, RGB camera, and thermal camera can improve the perception in day and night environments. To fuse the sensors, accurate extrinsic calibration is important. However, most existing methods use offline calibration, which is tedious, as it requires the use of special targets and human intervention. Meanwhile, among the online calibration methods, little attention has been paid to LiDAR and thermal camera online calibration. Thus, in this paper, an online extrinsic calibration method is proposed to solve the problem, called RGBDTCalibNet. It can achieve the extrinsic calibration between a 3D LiDAR and an RGB camera, as well as a 3D LiDAR and a thermal camera. It leverages CNN-based deep learning methods for feature extraction and feature matching. The proposed network only takes a pair of images and point cloud as the input and outputs the extrinsic parameters in a single shot. It is trained and tested on two datasets: KITTI360 and our own dataset collected in the NTU campus, using two types of LiDAR: Velodyne HDL-64 and Livox Horizon. Both demonstrate the effectiveness, robustness and accuracy of RGBDTCalibNet. The code will be open-sourced at https://github.com/sanatmharolkar/RGBDTCalibNet.git


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    RGBDTCalibNet: End-to-end Online Extrinsic Calibration between a 3D LiDAR, an RGB Camera and a Thermal Camera


    Contributors:
    Mharolkar, Sanat (author) / Zhang, Jun (author) / Peng, Guohao (author) / Liu, Yiyao (author) / Wang, Danwei (author)


    Publication date :

    2022-10-08


    Size :

    2980492 byte




    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English



    Extrinsic Calibration of a 3D-LIDAR and a Camera

    Mishra, Subodh / Pandey, Gaurav / Saripalli, Srikanth | IEEE | 2020


    LiDAR-Camera System Automatic Extrinsic Calibration in Rail Transit

    Wu, Qian / Zhang, Jin / Sheng, Jie et al. | IEEE | 2022


    A Survey of Extrinsic Calibration of LiDAR and Camera

    Wang, Yuru / Li, Jian / Sun, Yi et al. | TIBKAT | 2022


    Line Feature Based Extrinsic Calibration of LiDAR and Camera

    Jiang, Jingjing / Xue, Peixin / Chen, Shitao et al. | IEEE | 2018


    A Survey of Extrinsic Calibration of LiDAR and Camera

    Wang, Yuru / Li, Jian / Sun, Yi et al. | British Library Conference Proceedings | 2022