Multi-modal sensor fusion is important in the perception module of autonomous driving. Fusion methods on geometry-based and vision-based sensors, e.g. LiDAR and camera, have shown great performance. Existing fusion approaches focus on comprehensive scene understanding, integrating multi-modal representations to capture global semantics. However, we demonstrate that global semantics is insufficient in complex scenarios. For example, state representations with global semantics do not perceive emerging vehicles quite well, especially when they are only partially observed by sensors. Shared semantics, namely, the common information between multi-modal sensors, should also be considered. Therefore, we proposed SASF, a Semantic Aware multi-modal Sensor Fusion method, to enhance the effect of shared semantics on perception. We use Transformer to capture global semantics and adopt Mutual Information Maximization to enhance the shared semantics. Based on the enhanced representation, we develop a GRU waypoint prediction network for motion planning. We validate the efficacy of our model in urban settings using the CARLA driving simulator. Experimental results demonstrate that our approach achieves better driving performance in complex scenarios.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Semantic-Aware Multi-modal Sensor Fusion for Motion Planning in Autonomous Driving


    Additional title:

    Lect. Notes Electrical Eng.


    Contributors:
    Wu, Meiping (editor) / Niu, Yifeng (editor) / Gu, Mancang (editor) / Cheng, Jin (editor) / Chen, Keyu (author) / Sun, Shiliang (author)

    Conference:

    International Conference on Autonomous Unmanned Systems ; 2021 ; Changsha, China September 24, 2021 - September 26, 2021



    Publication date :

    2022-03-18


    Size :

    10 pages





    Type of media :

    Article/Chapter (Book)


    Type of material :

    Electronic Resource


    Language :

    English





    Semantic-Aware Multi-modal Sensor Fusion for Motion Planning in Autonomous Driving

    Chen, Keyu / Sun, Shiliang | British Library Conference Proceedings | 2022



    Uncertainty-Aware Motion Planning for Autonomous Driving on Highway

    Yang, Kai / Tang, Xiaolin / Wang, Ming et al. | IEEE | 2022