In this paper, a novel LiDAR-Camera Alignment (LCA) method using homogeneous local-global spatial aware representation is proposed. Compared with the state-of-the-art methods (e.g., LCCNet), our proposition holds 2 main superiorities. First, homogeneous multi-modality representation learned with a uniform CNN model is applied along the iterative prediction stages, instead of the state-of-the-art heterogeneous counterparts extracted from the separated modality-wise CNN models within each stage. In this way, the model size can be significantly decreased (e.g., 12.39M (ours) vs. 333.75M (LCCNet)). Meanwhile, within our proposition the interaction between LiDAR and camera data is built during feature learning to better exploit the descriptive clues, which has not been well concerned by the existing approaches. Secondly, we propose to equip the learned LCA representation with local-global spatial aware capacity via encoding CNN’s local convolutional features with Transformer’s non-local self-attention manner. Accordingly, the local fine details and global spatial context can be jointly captured by the encoded local features. And, they will be jointly used for LCA. On the other hand, the existing methods generally choose to reveal the global spatial property via intuitively concatenating the local features. Additionally at the initial LCA stage, LiDAR is roughly aligned with camera by our pre-alignment method, according to the point distribution characteristics of its 2D projection version with the initial extrinsic parameters. Although its structure is simple, it can essentially alleviate LCA’s difficulty for the consequent stages. To better optimize LCA, a novel loss function that builds the correlation between translation and rotation loss items is also proposed. The experiments on KITTI data verifies the superiority of our proposition both on effectiveness and efficiency. The source code will be released at https://github.com/Zaf233/Light-weight-LCA upon acceptance.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Lightweight LiDAR-Camera Alignment With Homogeneous Local-Global Aware Representation


    Contributors:
    Zhu, Angfan (author) / Xiao, Yang (author) / Liu, Chengxin (author) / Tan, Mingkui (author) / Cao, Zhiguo (author)

    Published in:

    Publication date :

    2024-11-01


    Size :

    3282623 byte




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English



    Robust lidar to camera sensor alignment

    OMAR YAHYA A / YU HAO / ZHANG WENDE | European Patent Office | 2024

    Free access

    A Comprehensive Approach for Camera/LIDAR Frame Alignment

    Bouain, Mokhtar / Berdjag, Denis / Fakhfakh, Nizar et al. | Springer Verlag | 2019


    EventAlign: LiDAR-Event Camera Calibration with Event Alignment Loss

    Dong, Pengfei / Hai, Renwei / Shen, Yanqing et al. | IEEE | 2024


    METHODS AND SYSTEMS FOR TWO-PHASE CAMERA TO LIDAR ALIGNMENT

    OMAR YOUSEF / WOLSKI MARK / YU HAO et al. | European Patent Office | 2023

    Free access

    GATED LOAD BALANCING FOR UNCERTAINTY AWARE CAMERA-LIDAR FUSION

    BALACHANDRAN BALAJI SHANKAR / RAVI KUMAR VARUN / YOGAMANI SENTHIL KUMAR | European Patent Office | 2025

    Free access