Sensor fusion is a crucial augmentation technique for improving the accuracy and reliability of perception sys-tems for automated vehicles under diverse driving conditions. However, adverse weather and low-light conditions remain challenging, where sensor performance degrades significantly, exposing vehicle safety to potential risks. Advanced sensors such as LiDARs can help mitigate the issue but with extremely high marginal costs. In this paper, we propose a novel transformer-based 3D object detection model “REDFormer” to tackle low visibility conditions, exploiting the power of a more practi-cal and cost-effective solution by leveraging bird's-eye-view camera-radar fusion. Using the nuScenes dataset with multi-radar point clouds, weather information, and time-of-day data, our model outperforms state-of-the-art (SOTA) models on clas-sification and detection accuracy. Finally, we provide extensive ablation studies of each model component on their contributions to address the above-mentioned challenges. Particularly, it is shown in the experiments that our model achieves a significant performance improvement over the baseline model in low-visibility scenarios, specifically exhibiting a 31.31% increase in rainy scenes and a 46.99% enhancement in nighttime scenes. The source code of this study is publicly available11https://github.com/PurdueDigitalTwin/REDFormer.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Radar Enlightens the Dark: Enhancing Low-Visibility Perception for Automated Vehicles with Camera-Radar Fusion


    Contributors:
    Cui, Can (author) / Ma, Yunsheng (author) / Lu, Juanwu (author) / Wang, Ziran (author)


    Publication date :

    2023-09-24


    Size :

    5154450 byte





    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English



    REDFormer: Radar Enlightens the Darkness of Camera Perception With Transformers

    Cui, Can / Ma, Yunsheng / Lu, Juanwu et al. | IEEE | 2024


    Radar and Camera Fusion based Moving Obstacle Tracking for Automated Vehicles

    Wang, Shihao / Ma, Zheng / Li, Ying et al. | IEEE | 2021


    Object Tracking System With Radar/Vision Fusion For Automated Vehicles

    SCHIFFMANN JAN K | European Patent Office | 2017

    Free access

    Object tracking system with radar/vision fusion for automated vehicles

    SCHIFFMANN JAN K | European Patent Office | 2020

    Free access

    CAMERA-RADAR FUSION USING CORRESPONDENCES

    MICHIELIN FRANCESCO / VOGEL OLIVER | European Patent Office | 2023

    Free access