This paper describes a multimodal vision sensor that integrates three types of cameras, including a stereo camera, a polarization camera and a panoramic camera. Each sensor provides a specific dimension of information: the stereo camera measures depth per pixel, the polarization obtains the degree of polarization, and the panoramic camera captures a 360° landscape. Data fusion and advanced environment perception could be built upon the combination of sensors. Designed especially for autonomous driving, this vision sensor is shipped with a robust semantic segmentation network. In addition, we demonstrate how cross-modal enhancement could be achieved by registering the color image and the polarization image. An example of water hazard detection is given. To prove the multimodal vision sensor’s compatibility with different devices, a brief runtime performance analysis is carried out.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    A multimodal vision sensor for autonomous driving


    Contributors:
    Sun, Dongming (author) / Huang, Xiao (author) / Yang, Kailun (author)

    Conference:

    Counterterrorism, Crime Fighting, Forensics, and Surveillance Technologies III ; 2019 ; Strasbourg,France


    Published in:

    Proc. SPIE ; 11166


    Publication date :

    2019-10-07





    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English



    Multimodal end-to-end autonomous driving

    Xiao, Yi / Codevilla Moraes, Felipe / Gurram, Akhil et al. | BASE | 2022

    Free access

    Multimodal End-to-End Autonomous Driving

    Xiao, Yi / Codevilla, Felipe / Gurram, Akhil et al. | IEEE | 2022


    Computer Vision for Autonomous Driving

    Kanchana, Bimsara / Peiris, Rojith / Perera, Damitha et al. | IEEE | 2021


    MULTIMODAL MOTION PLANNING FRAMEWORK FOR AUTONOMOUS DRIVING VEHICLES

    ZHANG YAJIA / LI DONG / ZHANG LIANGLIANG et al. | European Patent Office | 2022

    Free access

    Collaborative Autonomous Driving: Vision and Challenges

    Dong, Zheng / Shi, Weisong / Tong, Guangmo et al. | IEEE | 2020