Many learning-based methods estimate ego-motion using visual sensors. However, visual sensors are prone to intense lighting variations and textureless scenarios. 4D radar, an emerging automotive sensor, complements visual sensors effectively due to its robustness in adverse weather and lighting conditions. This paper presents an end-to-end 4D radar-visual odometry (4DRVO) approach that combines sparse point cloud data from 4D radar with image information from cameras. Using the Feature Pyramid, Pose Warping, and Cost Volume (PWC) network architecture, we extract 4D radar point features and image features at multiple scales. We then employ a hierarchical iterative refinement approach to supervise the estimated pose. We propose a novel Cross-Modal Transformer (CMT) module to effectively fuse the 4D radar point modality, image modality, and 4D radar point-image connection modality at multiple scales, achieving cross-modal feature interaction and multi-modal feature fusion. Additionally, we designed a point confidence estimation module to mitigate the impact of dynamic objects on odometry estimation. Extensive experiments were conducted on the View-of-Delft (VoD) dataset, showcasing the remarkable performance and effectiveness of the proposed 4D radar-visual odometry method.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    Deep 4D Automotive Radar-Camera Fusion Odometry with Cross-Modal Transformer Fusion


    Weitere Titelangaben:

    Sae Technical Papers


    Beteiligte:
    Zhuo, Guirong (Autor:in) / Xiong, Lu (Autor:in) / Zhou, Mingyu (Autor:in) / Lu, Shouyi (Autor:in) / Lu, Xinfei (Autor:in)

    Kongress:

    SAE 2023 Intelligent and Connected Vehicles Symposium ; 2023



    Erscheinungsdatum :

    2023-12-20




    Medientyp :

    Aufsatz (Konferenz)


    Format :

    Print


    Sprache :

    Englisch




    Multi-Camera Sensor Fusion for Visual Odometry using Deep Uncertainty Estimation

    Kaygusuz, Nimet / Mendez, Oscar / Bowden, Richard | IEEE | 2021


    Efficient Deep-Learning 4D Automotive Radar Odometry Method

    Lu, Shouyi / Zhuo, Guirong / Xiong, Lu et al. | IEEE | 2024


    LiDAR-Camera Fusion for Depth Enhanced Unsupervised Odometry

    Fetic, Naida / Aydemir, Eren / Unel, Mustafa | IEEE | 2022


    Automotive visual odometry

    Buczko, Martin / Shaker Verlag | TIBKAT | 2018


    CAMERA-RADAR FUSION USING CORRESPONDENCES

    MICHIELIN FRANCESCO / VOGEL OLIVER | Europäisches Patentamt | 2023

    Freier Zugriff