This paper presents a study on pedestrian classification based on deep learning using data from a monocular camera and a 3D LIDAR sensor, separately and in combination. Early and late multi-modal sensor fusion approaches are revisited and compared in terms of classification performance. The problem of pedestrian classification finds applications in advanced driver assistance system (ADAS) and autonomous driving, and it has regained particular attention recently because, among other reasons, safety involving self-driving vehicles. Convolutional Neural Networks (CNN) is used in this work as classifier in distinct situations: having a single sensor data as input, and by combining data from both sensors in the CNN input layer. Range (distance) and intensity (reflectance) data from LIDAR are considered as separate channels, where data from the LIDAR sensor is feed to the CNN in the form of dense maps, as the result of sensor coordinate transformation and spatial filtering; this allows a direct implementation of the same CNN-based approach on both sensors data. In terms of late-fusion, the outputs from individual CNNs are combined by means of learning and non-learning approaches. Pedestrian classification is evaluated on a ‘binary classification’ dataset created from the KITTI Vision Benchmark Suite, and results are shown for each sensor-modality individually, and for the fusion strategies.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    Multimodal CNN Pedestrian Classification: A Study on Combining LIDAR and Camera Data




    Erscheinungsdatum :

    01.11.2018


    Format / Umfang :

    1259283 byte





    Medientyp :

    Aufsatz (Konferenz)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch



    CNN-LIDAR pedestrian classification: combining range and reflectance data

    Melotti, Gledson / Asvadi, Alireza / Premebida, Cristiano | IEEE | 2018


    Reliable pedestrian recognition combining high-definition LIDAR and vision data

    Kidono, Kiyosumi / Naito, Takashi / Miura, Jun | IEEE | 2012


    GROUND SURFACE IMAGING COMBINING LIDAR AND CAMERA DATA

    KAGAN ASAF / BERMAN DANA / LEIBOVITZ GUY et al. | Europäisches Patentamt | 2021

    Freier Zugriff

    Ground surface imaging combining LiDAR and camera data

    KAGAN ASAF / BERMAN DANA / LEIBOVITZ GUY et al. | Europäisches Patentamt | 2023

    Freier Zugriff

    Multiview random forest of local experts combining RGB and LIDAR data for pedestrian detection

    Gonzalez, Alejandro / Villalonga, Gabriel / Xu, Jiaolong et al. | IEEE | 2015