Multi-modal fusion is a promising approach to boost the autonomous driving performance and has already received a large amount of attention. Meanwhile, to increase driving reliability under distinct scenarios, it is important to handle unforeseen weather events in the training dataset, which is known as an Out-Of-Distribution (OOD) problem, for autonomous driving algorithms. In this paper, we consider those two aspects and propose an end-to-end multi-modal domain-enhanced framework, namely CrossFuser, to meet the safety orientated driving requirements. CrossFuser first integrates both image and lidar modalities to generate a robust environmental representation through conjoint mapping, elastic disentanglement, and attention mechanism. Further, the perception embedding is used to calculate corresponding waypoints by a waypoint prediction network, consisting of Gate Recurrent Units (GRUs). Finally, the final control commands are calculated by low-level control functions. We conduct experiments on the Car Learning to Act (CARLA) driving simulator involving complex weather conditions under urban scenarios, the results show that CrossFuser can outperform the state of the art.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    CrossFuser: Multi-Modal Feature Fusion for End-to-End Autonomous Driving Under Unseen Weather Conditions


    Beteiligte:
    Wu, Weishang (Autor:in) / Deng, Xiaoheng (Autor:in) / Jiang, Ping (Autor:in) / Wan, Shaohua (Autor:in) / Guo, Yuanxiong (Autor:in)

    Erschienen in:

    Erscheinungsdatum :

    2023-12-01


    Format / Umfang :

    5779457 byte




    Medientyp :

    Aufsatz (Zeitschrift)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch



    Benchmarking Image Sensors Under Adverse Weather Conditions for Autonomous Driving

    Bijelic, Mario / Gruber, Tobias / Ritter, Werner | IEEE | 2018




    EVALUATING MULTI-MODAL TRAJECTORY PREDICTIONS FOR AUTONOMOUS DRIVING

    ETTINGER SCOTT MORGAN / ANGUELOV DRAGOMIR | Europäisches Patentamt | 2022

    Freier Zugriff

    Multi-sensor fusion for autonomous driving

    Zhang, Xinyu / Li, Jun / Li, Zhiwei et al. | TIBKAT | 2023