The dominant multi-camera 3D detection paradigm is based on explicit 3D feature construction, which requires complicated indexing of local image-view features via 3D-to-2D projection. Other methods implicitly introduce geometric positional encoding and perform global attention (e.g., PETR) to build the relationship between image tokens and 3D objects. The 3D-to-2D perspective inconsistency and global attention lead to a weak correlation between foreground tokens and queries, resulting in slow convergence. We propose Focal-PETR with instance-guided supervision and spatial alignment module to adaptively focus object queries on discriminative foreground regions. Focal-PETR additionally introduces a down-sampling strategy to reduce the consumption of global attention. Our model achieves leading performance on the large-scale nuScenes benchmark and a superior speed of 30 FPS on a single RTX3090 GPU. Extensive experiments show that our method outperforms PETR while consuming 3x fewer training hours. The code is made publicly available.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Focal-PETR: Embracing Foreground for Efficient Multi-Camera 3D Object Detection


    Contributors:
    Wang, Shihao (author) / Jiang, Xiaohui (author) / Li, Ying (author)

    Published in:

    Publication date :

    2024-01-01


    Size :

    2796075 byte




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English



    Foreground detection

    JALES COSTA BRUNO SIELLY / CORONA ENRIQUE / PUSKORIUS GINTARAS VINCENT et al. | European Patent Office | 2020

    Free access

    FOREGROUND DETECTION

    JALES COSTA BRUNO SIELLY / CORONA ENRIQUE / PUSKORIUS GINTARAS VINCENT et al. | European Patent Office | 2020

    Free access

    Foreground Object Detection by Motion-based Grouping of Object Parts

    Wang, Zhipeng / Cui, Jinshi / Zha, Hongbin et al. | Springer Verlag | 2014

    Free access


    Foreground Object Detection by Motion-based Grouping of Object Parts

    Wang, Zhipeng / Cui, Jinshi / Zha, Hongbin et al. | Springer Verlag | 2014

    Free access