Recently significant progress has been made in 3D detection. However, it is still challenging to detect small contour objects under complex scenes. This paper proposes a novel Attention-based Multi-phase Multi-task Fusion (AMMF) that uses point-level, RoI-level, and multi-task fusions to complement the disadvantages of LiDAR and camera, to solve this challenge. First, at the feature extraction phase, AMMF uses the Low and High-level Fusion with Matching Attention (LHF-MA) and efficient FPN (eFPN) to perform point-level fusion for cross sensors and single sensor, respectively. Instead of merging each level and using expensive 3D CNN like other methods, LHF-MA fuses low-level spatial location and high-level contextual feature of 2D CNN customized feature extractors and ignores the fusion of middle levels, reducing the computational cost. Then, at the proposal generation phase, Progressive Proposal Fusion (PPF) with learned attention map is used to perform coarse-to-fine RoI-level fusion, instead of only combining coarse-grained features at high-level of network. PPF using progressively increasing IoU thresholds could avoid overfitting and improve the performance. Note that the matching attentions and learned attention maps are utilized to weigh the priority of different sensors. Moreover, to solve the sparseness of point-wise fusion between LiDAR BEV and RGB image, AMMF uses multi-task fusion that generates pseudo-LiDAR from camera by depth estimation task, to guide this point-wise fusion. Finally, AMMF performs excellently for detecting small contour objects like pedestrians, cyclists, and distant cars. On the KITTI, AMMF finishes 3.62% improvements in the moderate instance for pedestrians. It achieves a 2.21% improvement in the >50 instance of LEVEL_2 level for vehicle on the Waymo Open Dataset. And AMMF is further verified on our customized dataset consisting of challenging scenarios like strong illumination and heavy shadow cases.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    AMMF: Attention-Based Multi-Phase Multi-Task Fusion for Small Contour Object 3D Detection


    Contributors:
    Xie, Bangquan (author) / Yang, Zongming (author) / Yang, Liang (author) / Wei, Ailin (author) / Weng, Xiaoxiong (author) / Li, Bing (author)


    Publication date :

    2023-02-01


    Size :

    10104666 byte




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English



    Attention‐based multi‐scale feature fusion for free‐space detection

    Pengfei Song / Hui Fan / Jinjiang Li et al. | DOAJ | 2022

    Free access

    Attention‐based multi‐scale feature fusion for free‐space detection

    Song, Pengfei / Fan, Hui / Li, Jinjiang et al. | Wiley | 2022

    Free access

    Attention in a Multi-Task Environment

    Andre, A. D. / Heers, S. T. | British Library Conference Proceedings | 1994


    Contour-based learning for object detection

    Shotton, J. / Blake, A. / Cipolla, R. | IEEE | 2005


    Contour-Based Learning for Object Detection

    Shotton, J. / Blake, A. / Cipolla, R. et al. | British Library Conference Proceedings | 2005