We present a novel compact deep multi-task learning model to handle various autonomous driving perception tasks in one forward pass. The model performs multiple views of semantic segmentation, depth estimation, light detection and ranging (LiDAR) segmentation, and bird’s eye view projection simultaneously without being supported by other models. We also provide an adaptive loss weighting algorithm to tackle the imbalanced learning issue that occurred due to plenty of given tasks. Through data pre-processing and intermediate sensor fusion techniques, the model can process and combine multiple input modalities retrieved from RGB cameras, dynamic vision sensors (DVS), and LiDAR placed at several positions on the ego vehicle. Therefore, a better understanding of a dynamically changing environment can be achieved. Based on the ablation study, the model variant trained with our proposed method achieves a better performance. Furthermore, a comparative study is also conducted to clarify its performance and effectiveness against the combination of some recent models. As a result, our model maintains better performance even with much fewer parameters. Hence, the model can inference faster with less GPU memory utilization. Moreover, the result tends to be consistent in 3 different CARLA simulation datasets and 1 real-world nuScenes-lidarseg dataset. To support future research, we share codes and other files publicly at https://github.com/oskarnatan/compact-perception.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Towards Compact Autonomous Driving Perception With Balanced Learning and Multi-Sensor Fusion


    Contributors:
    Natan, Oskar (author) / Miura, Jun (author)


    Publication date :

    2022-09-01


    Size :

    4336014 byte




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English



    Multi-sensor fusion for autonomous driving

    Zhang, Xinyu / Li, Jun / Li, Zhiwei et al. | TIBKAT | 2023


    Towards Autonomous Driving: a Multi-Modal 360° Perception Proposal

    Beltran, Jorge / Guindel, Carlos / Cortes, Irene et al. | IEEE | 2020


    Overview of Data Fusion in Autonomous Driving Perception

    Zhang, Xinyu / Li, Jun / Li, Zhiwei et al. | Springer Verlag | 2023


    Environmental Perception and Multi-Sensor Data Fusion for Off-Road Autonomous Vehicles

    Xiang, Z. / Ozguner, U. / IEEE | British Library Conference Proceedings | 2005


    Sensor fusion for autonomous driving transition control

    CHANDY ASHOK | European Patent Office | 2019

    Free access