In autonomous vehicles, understanding the surrounding 3D environment of the ego vehicle in real-time is essential. A compact way to represent scenes while encoding geometric distances and semantic object information is via 3D semantic occupancy maps. State of the art 3D mapping methods leverage transformers with cross-attention mechanisms to elevate 2D vision-centric camera features into the 3D domain. However, these methods encounter significant challenges in real-time applications due to their high computational demands during inference. This limitation is particularly problematic in autonomous vehicles, where GPU resources must be shared with other tasks such as localization and planning. In this paper, we introduce an approach that extracts features from front-view 2D camera images and LiDAR scans, then employs a sparse convolution network (Minkowski Engine), for 3D semantic occupancy prediction. Given that outdoor scenes in autonomous driving scenarios are inherently sparse, the utilization of sparse convolution is particularly apt. By jointly solving the problems of 3D scene completion of sparse scenes and 3D semantic segmentation, we provide a more efficient learning framework suitable for real-time applications in autonomous vehicles. We also demonstrate competitive accuracy on the nuScenes dataset.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Real-time 3D semantic occupancy prediction for autonomous vehicles using memory-efficient sparse convolution


    Contributors:
    Sze, Samuel (author) / Kunze, Lars (author)


    Publication date :

    2024-06-02


    Size :

    1996668 byte





    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English



    Real-time Occupancy Mapping System for Autonomous Vehicles

    MISHRA LALAN JEE / WIETFELDT RICHARD DOMINIC / CZOMPO JOSEPH | European Patent Office | 2016

    Free access

    Puddle occupancy grid for autonomous vehicles

    WANG YUCHI / JAMES MICHAEL / CHEN ROBERT et al. | European Patent Office | 2024

    Free access

    Puddle occupancy grid for autonomous vehicles

    WANG YUCHI / JAMES MICHAEL / CHEN ROBERT et al. | European Patent Office | 2023

    Free access

    PUDDLE OCCUPANCY GRID FOR AUTONOMOUS VEHICLES

    WANG YUCHI / JAMES MICHAEL / CHEN ROBERT et al. | European Patent Office | 2022

    Free access