Autonomous vehicles are equipped with a multi-modal sensor setup to enable the car to drive safely. The initial calibration of such perception sensors is a highly matured topic and is routinely done in an automated factory environment. However, an intriguing question arises on how to maintain the calibration quality throughout the vehicle’s operating duration. Another challenge is to calibrate multiple sensors jointly to ensure no propagation of systemic errors. In this paper, we propose Camera Lidar Calibration Network (CaLiCaNet), an end-to-end deep self-calibration network which addresses the automatic calibration problem for pinhole camera and Lidar. We jointly predict the camera intrinsic parameters (focal length and distortion) as well as Lidar-Camera extrinsic parameters (rotation and translation), by regressing feature correlation between the camera image and the Lidar point cloud. The network is arranged in a Siamese-twin structure to constrain the network features learning to a mutually shared feature in both point cloud and camera (Lidar-camera constraint). Evaluation using KITTI datasets shows that we achieve 0.154° and 0.059 m accuracy with a reprojection error of 0.028 pixel with a single-pass inference. We also provide an ablative study of how our end-to-end learning architecture offers lower terminal loss (21% decrease in rotation loss) compared to isolated calibration.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    End-to-End Lidar-Camera Self-Calibration for Autonomous Vehicles


    Contributors:


    Publication date :

    2023-06-04


    Size :

    1791823 byte





    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English



    LIDAR-to-camera transformation during sensor calibration for autonomous vehicles

    ZHANG ZHENGYU / YANG LIN / WHEELER MARK DAMON | European Patent Office | 2024

    Free access

    LiDAR eyes for autonomous vehicles

    Thakker, T. / Pulikkaseril, C. / Lam, S. et al. | SPIE | 2019


    Panoptic Based Camera and Lidar Fusion for Distance Estimation in Autonomous Driving Vehicles

    Jose, Edwin / P, Aparna M / Patil, Mrinalini et al. | British Library Conference Proceedings | 2022


    Panoptic Based Camera and Lidar Fusion for Distance Estimation in Autonomous Driving Vehicles

    Jose, Edwin / P, Aparna M / Patil, Mrinalini et al. | British Library Conference Proceedings | 2022


    Self-Calibration of Multiple LiDARs for Autonomous Vehicles

    Zhang, Zherui / Fu, Chen / Dong, Chiyu et al. | IEEE | 2021