Multi-modal sensor fusion is important in the perception module of autonomous driving. Fusion methods on geometry-based and vision-based sensors, e.g. LiDAR and camera, have shown great performance. Existing fusion approaches focus on comprehensive scene understanding, integrating multi-modal representations to capture global semantics. However, we demonstrate that global semantics is insufficient in complex scenarios. For example, state representations with global semantics do not perceive emerging vehicles quite well, especially when they are only partially observed by sensors. Shared semantics, namely, the common information between multi-modal sensors, should also be considered. Therefore, we proposed SASF, a Semantic Aware multi-modal Sensor Fusion method, to enhance the effect of shared semantics on perception. We use Transformer to capture global semantics and adopt Mutual Information Maximization to enhance the shared semantics. Based on the enhanced representation, we develop a GRU waypoint prediction network for motion planning. We validate the efficacy of our model in urban settings using the CARLA driving simulator. Experimental results demonstrate that our approach achieves better driving performance in complex scenarios.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    Semantic-Aware Multi-modal Sensor Fusion for Motion Planning in Autonomous Driving


    Weitere Titelangaben:

    Lect. Notes Electrical Eng.


    Beteiligte:
    Wu, Meiping (Herausgeber:in) / Niu, Yifeng (Herausgeber:in) / Gu, Mancang (Herausgeber:in) / Cheng, Jin (Herausgeber:in) / Chen, Keyu (Autor:in) / Sun, Shiliang (Autor:in)

    Kongress:

    International Conference on Autonomous Unmanned Systems ; 2021 ; Changsha, China September 24, 2021 - September 26, 2021



    Erscheinungsdatum :

    18.03.2022


    Format / Umfang :

    10 pages





    Medientyp :

    Aufsatz/Kapitel (Buch)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch





    Semantic-Aware Multi-modal Sensor Fusion for Motion Planning in Autonomous Driving

    Chen, Keyu / Sun, Shiliang | British Library Conference Proceedings | 2022



    Uncertainty-Aware Motion Planning for Autonomous Driving on Highway

    Yang, Kai / Tang, Xiaolin / Wang, Ming et al. | IEEE | 2022