3D multi-object tracking is a crucial component in the perception system of autonomous driving vehicles. Tracking all dynamic objects around the vehicle is essential for tasks such as obstacle avoidance and path planning. Autonomous vehicles are usually equipped with different sensor modalities to improve accuracy and reliability. While sensor fusion has been widely used in object detection networks in recent years, most existing multi-object tracking algorithms either rely on a single input modality, or do not fully exploit the information provided by multiple sensing modalities. In this work, we propose an end-to-end network for joint object detection and tracking based on radar and camera sensor fusion. Our proposed method uses a center-based radar-camera fusion algorithm for object detection and utilizes a greedy algorithm for object association. The proposed greedy algorithm uses the depth, velocity and 2D displacement of the detected objects to associate them through time. This makes our tracking algorithm very robust to occluded and overlapping objects, as the depth and velocity information can help the network in distinguishing them. We evaluate our method on the challenging nuScenes dataset, where it achieves 20.0 AMOTA and outperforms all vision-based 3D tracking methods in the benchmark, as well as the baseline LiDAR-based method. Our method is online with a runtime of 35ms per image, making it very suitable for autonomous driving applications.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    CFTrack: Center-based Radar and Camera Fusion for 3D Multi-Object Tracking


    Contributors:


    Publication date :

    2021-07-11


    Size :

    1899475 byte




    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English



    3D Multi-Object Tracking Based on Radar-Camera Fusion

    Lin, Zihao / Hu, Jianming | IEEE | 2022


    Multiple Camera Fusion for Multi-Object Tracking

    Dockstader, S. L. / Tekalp, A. M. / IEEE Computer Society | British Library Conference Proceedings | 2001


    Multiple camera fusion for multi-object tracking

    Dockstader, S.L. / Tekalp, A.M. | IEEE | 2001


    Optimization based 3D Multi-Object Tracking using Camera and Radar Data

    Poschmann, Johannes / Pfeifer, Tim / Protzel, Peter | IEEE | 2021