In this paper, a multi-modal 360° framework for 3D object detection and tracking for autonomous vehicles is presented. The process is divided into four main stages. First, images are fed into a CNN network to obtain instance segmentation of the surrounding road participants. Second, LiDAR-to-image association is performed for the estimated mask proposals. Then, the isolated points of every object are processed by a PointNet ensemble to compute their corresponding 3D bounding boxes and poses. Lastly, a tracking stage based on Unscented Kalman Filter is used to track the agents along time. The solution, based on a novel sensor fusion configuration, provides accurate and reliable road environment detection. A wide variety of tests of the system, deployed in an autonomous vehicle, have successfully assessed the suitability of the proposed perception stack in a real autonomous driving application.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Towards Autonomous Driving: a Multi-Modal 360° Perception Proposal




    Publication date :

    2020-09-20


    Size :

    3122329 byte




    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English



    UNISOURCE MULTI-MODAL PERCEPTION IN AUTONOMOUS VEHICLES

    LI JIAN / SU HAN | European Patent Office | 2022

    Free access

    SYSTEMS AND METHODS FOR MULTI-MODAL DATA AUGMENTATION FOR PERCEPTION TASKS IN AUTONOMOUS DRIVING

    ZHONG YIQI / HUANG XINYU / GUO YULIANG et al. | European Patent Office | 2023

    Free access


    EVALUATING MULTI-MODAL TRAJECTORY PREDICTIONS FOR AUTONOMOUS DRIVING

    ETTINGER SCOTT MORGAN / ANGUELOV DRAGOMIR | European Patent Office | 2022

    Free access

    PERCEPTION FIELDS FOR AUTONOMOUS DRIVING

    RAICHELGAUZ IGAL / ENGELSOY JULIUS | European Patent Office | 2024

    Free access