In this paper, a novel LiDAR-Camera Alignment (LCA) method using homogeneous local-global spatial aware representation is proposed. Compared with the state-of-the-art methods (e.g., LCCNet), our proposition holds 2 main superiorities. First, homogeneous multi-modality representation learned with a uniform CNN model is applied along the iterative prediction stages, instead of the state-of-the-art heterogeneous counterparts extracted from the separated modality-wise CNN models within each stage. In this way, the model size can be significantly decreased (e.g., 12.39M (ours) vs. 333.75M (LCCNet)). Meanwhile, within our proposition the interaction between LiDAR and camera data is built during feature learning to better exploit the descriptive clues, which has not been well concerned by the existing approaches. Secondly, we propose to equip the learned LCA representation with local-global spatial aware capacity via encoding CNN’s local convolutional features with Transformer’s non-local self-attention manner. Accordingly, the local fine details and global spatial context can be jointly captured by the encoded local features. And, they will be jointly used for LCA. On the other hand, the existing methods generally choose to reveal the global spatial property via intuitively concatenating the local features. Additionally at the initial LCA stage, LiDAR is roughly aligned with camera by our pre-alignment method, according to the point distribution characteristics of its 2D projection version with the initial extrinsic parameters. Although its structure is simple, it can essentially alleviate LCA’s difficulty for the consequent stages. To better optimize LCA, a novel loss function that builds the correlation between translation and rotation loss items is also proposed. The experiments on KITTI data verifies the superiority of our proposition both on effectiveness and efficiency. The source code will be released at https://github.com/Zaf233/Light-weight-LCA upon acceptance.
Lightweight LiDAR-Camera Alignment With Homogeneous Local-Global Aware Representation
IEEE Transactions on Intelligent Transportation Systems ; 25 , 11 ; 15922-15933
2024-11-01
3282623 byte
Article (Journal)
Electronic Resource
English
A Comprehensive Approach for Camera/LIDAR Frame Alignment
Springer Verlag | 2019
|METHODS AND SYSTEMS FOR TWO-PHASE CAMERA TO LIDAR ALIGNMENT
European Patent Office | 2023
|GATED LOAD BALANCING FOR UNCERTAINTY AWARE CAMERA-LIDAR FUSION
European Patent Office | 2025
|