Multi-modal sensor fusion is important in the perception module of autonomous driving. Fusion methods on geometry-based and vision-based sensors, e.g. LiDAR and camera, have shown great performance. Existing fusion approaches focus on comprehensive scene understanding, integrating multi-modal representations to capture global semantics. However, we demonstrate that global semantics is insufficient in complex scenarios. For example, state representations with global semantics do not perceive emerging vehicles quite well, especially when they are only partially observed by sensors. Shared semantics, namely, the common information between multi-modal sensors, should also be considered. Therefore, we proposed SASF, a Semantic Aware multi-modal Sensor Fusion method, to enhance the effect of shared semantics on perception. We use Transformer to capture global semantics and adopt Mutual Information Maximization to enhance the shared semantics. Based on the enhanced representation, we develop a GRU waypoint prediction network for motion planning. We validate the efficacy of our model in urban settings using the CARLA driving simulator. Experimental results demonstrate that our approach achieves better driving performance in complex scenarios.
Semantic-Aware Multi-modal Sensor Fusion for Motion Planning in Autonomous Driving
Lect. Notes Electrical Eng.
International Conference on Autonomous Unmanned Systems ; 2021 ; Changsha, China September 24, 2021 - September 26, 2021
Proceedings of 2021 International Conference on Autonomous Unmanned Systems (ICAUS 2021) ; Kapitel : 111 ; 1117-1126
2022-03-18
10 pages
Aufsatz/Kapitel (Buch)
Elektronische Ressource
Englisch
Multi-sensor fusion for autonomous driving
TIBKAT | 2023
|PLANNING-AWARE PREDICTION FOR CONTROL-AWARE AUTONOMOUS DRIVING MODULES
Europäisches Patentamt | 2023
|