To achieve more accurate perception performance, LiDAR and camera are gradually chosen to improve 3D object detection simultaneously. However, it is still a non-trivial task to build an effective fusion mechanism, and this is hindering the development of multi-modal based method. Especially, the mapping relationship construction between two modalities is far from fully explored. Canonical cross-modal mapping suffers from failure when the calibration matrix is incorrect, and it also greatly wastes the amount and density of RGB image information. This paper aims to extend the traditional one-to-one alignment relationship between LiDAR and camera. For all projected point clouds, we enhance their cross-modal mapping relationship through aggregating color-texture related feature and shape-contour related feature. Further, a mapping pyramid is proposed to leverage the semantic representation of the image feature at different stages. Based on the above mapping enhancement strategies, our method increases the engagement rate of image. Finally, we design a fusion module based on an attention mechanism to improve the point cloud feature with the auxiliary image feature. Extensive experiments on the KITTI dataset and SUN-RGBD dataset show that our model achieves satisfactory 3D object detection, especially for categories with sparse point clouds compared with other multi-modal fusion networks.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    MENet: Multi-Modal Mapping Enhancement Network for 3D Object Detection in Autonomous Driving


    Beteiligte:
    Liu, Moyun (Autor:in) / Chen, Youping (Autor:in) / Xie, Jingming (Autor:in) / Zhu, Yijie (Autor:in) / Zhang, Yang (Autor:in) / Yao, Lei (Autor:in) / Bing, Zhenshan (Autor:in) / Zhuang, Genghang (Autor:in) / Huang, Kai (Autor:in) / Zhou, Joey Tianyi (Autor:in)


    Erscheinungsdatum :

    01.08.2024


    Format / Umfang :

    3951059 byte




    Medientyp :

    Aufsatz (Zeitschrift)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch



    LEVERAGING UNCERTAINTIES FOR DEEP MULTI-MODAL OBJECT DETECTION IN AUTONOMOUS DRIVING

    Feng, Di / Cao, Yifan / Rosenbaum, Lars et al. | British Library Conference Proceedings | 2020


    Leveraging Uncertainties for Deep Multi-modal Object Detection in Autonomous Driving

    Feng, Di / Cao, Yifan / Rosenbaum, Lars et al. | IEEE | 2020


    Multi-Modal 3D Object Detection in Autonomous Driving: A Survey and Taxonomy

    Wang, Li / Zhang, Xinyu / Song, Ziying et al. | IEEE | 2023


    MENeT: a monitoring and protocol analysis tool for LAN

    Junejo, N. / Junejo, N.A. / Unar, M.A. | IEEE | 2004