Multi-modal sensor fusion is important in the perception module of autonomous driving. Fusion methods on geometry-based and vision-based sensors, e.g. LiDAR and camera, have shown great performance. Existing fusion approaches focus on comprehensive scene understanding, integrating multi-modal representations to capture global semantics. However, we demonstrate that global semantics is insufficient in complex scenarios. For example, state representations with global semantics do not perceive emerging vehicles quite well, especially when they are only partially observed by sensors. Shared semantics, namely, the common information between multi-modal sensors, should also be considered. Therefore, we proposed SASF, a Semantic Aware multi-modal Sensor Fusion method, to enhance the effect of shared semantics on perception. We use Transformer to capture global semantics and adopt Mutual Information Maximization to enhance the shared semantics. Based on the enhanced representation, we develop a GRU waypoint prediction network for motion planning. We validate the efficacy of our model in urban settings using the CARLA driving simulator. Experimental results demonstrate that our approach achieves better driving performance in complex scenarios.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    Semantic-Aware Multi-modal Sensor Fusion for Motion Planning in Autonomous Driving


    Weitere Titelangaben:

    Lect. Notes Electrical Eng.


    Beteiligte:
    Wu, Meiping (Herausgeber:in) / Niu, Yifeng (Herausgeber:in) / Gu, Mancang (Herausgeber:in) / Cheng, Jin (Herausgeber:in) / Chen, Keyu (Autor:in) / Sun, Shiliang (Autor:in)

    Kongress:

    International Conference on Autonomous Unmanned Systems ; 2021 ; Changsha, China September 24, 2021 - September 26, 2021



    Erscheinungsdatum :

    2022-03-18


    Format / Umfang :

    10 pages





    Medientyp :

    Aufsatz/Kapitel (Buch)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch





    Uncertainty-Aware Motion Planning for Autonomous Driving on Highway

    Yang, Kai / Tang, Xiaolin / Wang, Ming et al. | IEEE | 2022


    Multi-sensor fusion for autonomous driving

    Zhang, Xinyu / Li, Jun / Li, Zhiwei et al. | TIBKAT | 2023


    PLANNING-AWARE PREDICTION FOR CONTROL-AWARE AUTONOMOUS DRIVING MODULES

    MCALLISTER ROWAN THOMAS / WULFE BLAKE WARREN / MERCAT JEAN et al. | Europäisches Patentamt | 2023

    Freier Zugriff

    Multi-Modal Sensor Fusion and Object Tracking for Autonomous Racing

    Karle, Phillip / Fent, Felix / Huch, Sebastian et al. | IEEE | 2023