Multi-vehicle cooperative perception has recently emerged for facilitating long-range and large-scale perception ability of connected automated vehicles (CAVs). Nonetheless, enormous efforts formulate collaborative perception as LiDAR-only 3D detection paradigm, neglecting the significance and complementary of dense image. In this work, we construct the first multi-modal vehicle-to-vehicle cooperative perception framework dubbed as V2VFormer++, where individual camera-LiDAR representation is incorporated with dynamic channel fusion (DCF) at bird’s-eye-view (BEV) space and ego-centric BEV maps from adjacent vehicles are aggregated by global-local transformer module. Specifically, channel-token mixer (CTM) with MLP design is developed to capture global response among neighboring CAVs, and position-aware fusion (PAF) further investigate the spatial correlation between each ego-networked map in a local perspective. In this manner, we could strategically determine which CAVs are desirable for collaboration and how to aggregate the foremost information from them. Quantitative and qualitative experiments are conducted on both publicly-available OPV2V and V2X-Sim 2.0 benchmarks, and our proposed V2VFormer++ reports the state-of-the-art cooperative perception performance, demonstrating its effectiveness and advancement. Moreover, ablation study and visualization analysis further suggest the strong robustness against diverse disturbances from real-world scenarios.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    V2VFormer++: Multi-Modal Vehicle-to-Vehicle Cooperative Perception via Global-Local Transformer


    Beteiligte:
    Yin, Hongbo (Autor:in) / Tian, Daxin (Autor:in) / Lin, Chunmian (Autor:in) / Duan, Xuting (Autor:in) / Zhou, Jianshan (Autor:in) / Zhao, Dezong (Autor:in) / Cao, Dongpu (Autor:in)


    Erscheinungsdatum :

    2024-02-01


    Format / Umfang :

    5031717 byte




    Medientyp :

    Aufsatz (Zeitschrift)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch



    V2VFormer: Vehicle-to-Vehicle Cooperative Perception With Spatial-Channel Transformer

    Lin, Chunmian / Tian, Daxin / Duan, Xuting et al. | IEEE | 2024


    V2X-ViT: Vehicle-to-Everything Cooperative Perception with Vision Transformer

    Xu, Runsheng / Xiang, Hao / Tu, Zhengzhong et al. | British Library Conference Proceedings | 2022


    Multi-modal vehicle

    NARENDRA HIRALAL KARADIA | Europäisches Patentamt | 2022

    Freier Zugriff

    MULTI-MODAL VEHICLE

    KARADIA NARENDRA H | Europäisches Patentamt | 2022

    Freier Zugriff

    MULTI-MODAL VEHICLE

    KARADIA NARENDRA HIRALAL | Europäisches Patentamt | 2020

    Freier Zugriff