We present a novel compact deep multi-task learning model to handle various autonomous driving perception tasks in one forward pass. The model performs multiple views of semantic segmentation, depth estimation, light detection and ranging (LiDAR) segmentation, and bird’s eye view projection simultaneously without being supported by other models. We also provide an adaptive loss weighting algorithm to tackle the imbalanced learning issue that occurred due to plenty of given tasks. Through data pre-processing and intermediate sensor fusion techniques, the model can process and combine multiple input modalities retrieved from RGB cameras, dynamic vision sensors (DVS), and LiDAR placed at several positions on the ego vehicle. Therefore, a better understanding of a dynamically changing environment can be achieved. Based on the ablation study, the model variant trained with our proposed method achieves a better performance. Furthermore, a comparative study is also conducted to clarify its performance and effectiveness against the combination of some recent models. As a result, our model maintains better performance even with much fewer parameters. Hence, the model can inference faster with less GPU memory utilization. Moreover, the result tends to be consistent in 3 different CARLA simulation datasets and 1 real-world nuScenes-lidarseg dataset. To support future research, we share codes and other files publicly at https://github.com/oskarnatan/compact-perception.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    Towards Compact Autonomous Driving Perception With Balanced Learning and Multi-Sensor Fusion


    Beteiligte:
    Natan, Oskar (Autor:in) / Miura, Jun (Autor:in)

    Erschienen in:

    Erscheinungsdatum :

    01.09.2022


    Format / Umfang :

    4336014 byte




    Medientyp :

    Aufsatz (Zeitschrift)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch



    Multi-sensor fusion for autonomous driving

    Zhang, Xinyu / Li, Jun / Li, Zhiwei et al. | TIBKAT | 2023


    Towards Autonomous Driving: a Multi-Modal 360° Perception Proposal

    Beltran, Jorge / Guindel, Carlos / Cortes, Irene et al. | IEEE | 2020


    Overview of Data Fusion in Autonomous Driving Perception

    Zhang, Xinyu / Li, Jun / Li, Zhiwei et al. | Springer Verlag | 2023


    Multi-sensor fusion 3D object detection for autonomous driving

    Alaba, Simegnew Yihunie / Ball, John E. | British Library Conference Proceedings | 2023


    Autonomous driving trolley device based on multi-sensor fusion slam

    ZHONG HONGLONG / JIANG HONG / ZHANG XIANGFENG et al. | Europäisches Patentamt | 2024

    Freier Zugriff