This paper presents a study on pedestrian classification based on deep learning using data from a monocular camera and a 3D LIDAR sensor, separately and in combination. Early and late multi-modal sensor fusion approaches are revisited and compared in terms of classification performance. The problem of pedestrian classification finds applications in advanced driver assistance system (ADAS) and autonomous driving, and it has regained particular attention recently because, among other reasons, safety involving self-driving vehicles. Convolutional Neural Networks (CNN) is used in this work as classifier in distinct situations: having a single sensor data as input, and by combining data from both sensors in the CNN input layer. Range (distance) and intensity (reflectance) data from LIDAR are considered as separate channels, where data from the LIDAR sensor is feed to the CNN in the form of dense maps, as the result of sensor coordinate transformation and spatial filtering; this allows a direct implementation of the same CNN-based approach on both sensors data. In terms of late-fusion, the outputs from individual CNNs are combined by means of learning and non-learning approaches. Pedestrian classification is evaluated on a ‘binary classification’ dataset created from the KITTI Vision Benchmark Suite, and results are shown for each sensor-modality individually, and for the fusion strategies.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Multimodal CNN Pedestrian Classification: A Study on Combining LIDAR and Camera Data




    Publication date :

    2018-11-01


    Size :

    1259283 byte





    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English



    CNN-LIDAR pedestrian classification: combining range and reflectance data

    Melotti, Gledson / Asvadi, Alireza / Premebida, Cristiano | IEEE | 2018


    Reliable pedestrian recognition combining high-definition LIDAR and vision data

    Kidono, Kiyosumi / Naito, Takashi / Miura, Jun | IEEE | 2012


    GROUND SURFACE IMAGING COMBINING LIDAR AND CAMERA DATA

    KAGAN ASAF / BERMAN DANA / LEIBOVITZ GUY et al. | European Patent Office | 2021

    Free access

    Ground surface imaging combining LiDAR and camera data

    KAGAN ASAF / BERMAN DANA / LEIBOVITZ GUY et al. | European Patent Office | 2023

    Free access

    Multiview random forest of local experts combining RGB and LIDAR data for pedestrian detection

    Gonzalez, Alejandro / Villalonga, Gabriel / Xu, Jiaolong et al. | IEEE | 2015