Recent advancements in perception for autonomous driving are driven by deep learning. In order to achieve robust and accurate scene understanding, autonomous vehicles are usually equipped with different sensors (e.g. cameras, LiDARs, Radars), and multiple sensing modalities can be fused to exploit their complementary properties. In this context, many methods have been proposed for deep multi-modal perception problems. However, there is no general guideline for network architecture design, and questions of “what to fuse”, “when to fuse”, and “how to fuse” remain open. This review paper attempts to systematically summarize methodologies and discuss challenges for deep multi-modal object detection and semantic segmentation in autonomous driving. To this end, we first provide an overview of on-board sensors on test vehicles, open datasets, and background information for object detection and semantic segmentation in autonomous driving research. We then summarize the fusion methodologies and discuss challenges and open questions. In the appendix, we provide tables that summarize topics and methods. We also provide an interactive online platform to navigate each reference: https://boschresearch.github.io/multimodalperception/.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Deep Multi-Modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges


    Contributors:


    Publication date :

    2021-03-01


    Size :

    3354193 byte




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English



    LEVERAGING UNCERTAINTIES FOR DEEP MULTI-MODAL OBJECT DETECTION IN AUTONOMOUS DRIVING

    Feng, Di / Cao, Yifan / Rosenbaum, Lars et al. | British Library Conference Proceedings | 2020


    Leveraging Uncertainties for Deep Multi-modal Object Detection in Autonomous Driving

    Feng, Di / Cao, Yifan / Rosenbaum, Lars et al. | IEEE | 2020



    Multi-Modal 3D Object Detection in Autonomous Driving: A Survey and Taxonomy

    Wang, Li / Zhang, Xinyu / Song, Ziying et al. | IEEE | 2023


    Deep semantic segmentation for automated driving: Taxonomy, roadmap and challenges

    Siam, Mennatullah / Elkerdawy, Sara / Jagersand, Martin et al. | IEEE | 2017