In autonomous driving, addressing occlusion scenarios is crucial yet challenging. Robust surrounding perception is essential for handling occlusions and aiding navigation. State-of-the-art models fuse LiDAR and Camera data to produce impressive perception results, but detecting occluded objects remains challenging. In this paper, we emphasize the crucial role of temporal cues in reinforcing resilience against occlusions in the bird’s eye view (BEV) semantic grid segmentation task. We proposed a novel architecture that enables the processing of temporal multi-step inputs, where the input at each time step comprises the spatial information encoded from fusing LiDAR and camera sensor readings. We experimented on the real-world nuScenes dataset and our results outperformed other baselines, with particularly large differences when evaluating on occluded and partially-occluded vehicles. Additionally, we applied the proposed model to downstream tasks, such as multi-step BEV prediction and trajectory forecasting of the ego-vehicle. The qualitative results obtained from these tasks underscore the adaptability and effectiveness of our proposed approach.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    TLCFuse: Temporal Multi-Modality Fusion Towards Occlusion-Aware Semantic Segmentation




    Publication date :

    2024-06-02


    Size :

    1783795 byte





    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English



    Hyperbolic Uncertainty Aware Semantic Segmentation

    Chen, Bike / Peng, Wei / Cao, Xiaofeng et al. | IEEE | 2024


    OCCLUSION AWARE SENSOR FUSION FOR EARLY CROSSING PEDESTRIAN DETECTION

    Palffy, Andras / Kooij, Julian F. P. / Gavrila, Dariu M. | British Library Conference Proceedings | 2019


    Occlusion aware sensor fusion for early crossing pedestrian detection

    Palffy, Andras / Kooij, Julian F. P. / Gavrila, Dariu M. | IEEE | 2019


    Machine Learning Multi-Modality Fusion Approaches Outperform Single-Modality & Traditional Approaches

    Garagic, Denis / Pelgrift, Daniel / Peskoe, Jacob et al. | IEEE | 2021


    OCCLUSION AWARE PLANNING

    SILVA WILLIAM ANTHONY / ANGUELOV DRAGOMIR DIMITROV / ZWIEBEL BENJAMIN ISAAC et al. | European Patent Office | 2019

    Free access