Multiple Object Tracking (MOT) is a significant task in autonomous driving. Nonetheless, relying on one single sensor is not robust enough, because one modality tends to fail in some challenging situations. Texture information from RGB cameras and 3D structure information from Light Detection and Ranging (LiDAR) have respective advantages under different circumstances. Therefore, feature fusion from multiple modalities contributes to the learning of discriminative features. However, it is nontrivial to achieve effective feature fusion due to the completely distinct information modality. Previous fusion methods usually fuse the top-level features after the backbones extract the features from different modalities. The feature fusion happens solely once, which limits the information interaction between different modalities. In this paper, we propose multi-scale interactive query and fusion between pixel-wise and point-wise features to obtain more discriminative features. In addition, an attention mechanism is utilized to conduct soft feature fusion between multiple pixels and points to avoid inaccurate match problems of previous single pixel-point fusion methods. We introduce PointNet++ to obtain multi-scale deep representations of point clouds and make it adaptive to our proposed interactive feature fusion between multi-scale features of images and point clouds. Through the interaction module, each modality can integrate more complementary information from the other modality. Besides, we explore the effectiveness of pre-training on each single modality and fine-tuning on the fusion-based model. Our method can achieve 90.32% MOTA and 72.44% HOTA on the KITTI benchmark and outperform other approaches without using multi-scale soft feature fusion.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    Interactive Multi-Scale Fusion of 2D and 3D Features for Multi-Object Vehicle Tracking


    Beteiligte:
    Wang, Guangming (Autor:in) / Peng, Chensheng (Autor:in) / Gu, Yingying (Autor:in) / Zhang, Jinpeng (Autor:in) / Wang, Hesheng (Autor:in)

    Erschienen in:

    Erscheinungsdatum :

    2023-10-01


    Format / Umfang :

    3243172 byte




    Medientyp :

    Aufsatz (Zeitschrift)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch



    Vehicle-Road Multi-View Interactive Data Fusion

    Zhang, Xinyu / Li, Jun / Li, Zhiwei et al. | Springer Verlag | 2023


    Multiple camera fusion for multi-object tracking

    Dockstader, S.L. / Tekalp, A.M. | IEEE | 2001


    Multi-Scan Sensor Fusion for Object Tracking

    JIA BIN / WANG XIAOHUI | Europäisches Patentamt | 2023

    Freier Zugriff

    Multiple Camera Fusion for Multi-Object Tracking

    Dockstader, S. L. / Tekalp, A. M. / IEEE Computer Society | British Library Conference Proceedings | 2001