Mulimodal learning is the process of reasoning with different types of data simultaneously using a neural network. This is no easy feat. Challenges to tackle include mapping features from one modality to another and finding a joint representation. As applied to connected vehicles, we propose a multimodal module that detects manipulations in the image sensor through verifying its consistency with the LiDAR sensor.A state-of-the-art multimodal network, Regnet, is adapted and trained for attack detection. Adaptation includes replacing the input layer and abstracting the multimodal input process. The KITTI dataset is extended to represent two attacks on connected vehicles: inpainting and translation. The end product is a smart multimodal module that abstracts connected-vehicle sensor input and guards against data integrity breaches.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Multi-Modal Deep Learning for Vehicle Sensor Data Abstraction and Attack Detection


    Contributors:


    Publication date :

    2019-09-01


    Size :

    739515 byte





    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English



    Multi-modal corner vehicle sensor module

    TENSELIUS DANIEL / CLAEZON FREDRICH / JOHANSSON MARTIN et al. | European Patent Office | 2023

    Free access

    Object velocity detection from multi-modal sensor data

    PRADHAN SABEEK MANI / SLOAN COOPER STOKES | European Patent Office | 2023

    Free access

    Multi-modal, multi-technology vehicle signal detection

    KROISOW CHRISTIAN / THUREMELA DAVID / YU NA et al. | European Patent Office | 2022

    Free access

    Deep learning based multi-modal perception and semi-automatic labelling algorithms for automotive sensor data

    Haase-Schütz, Christian / kassel university press | TIBKAT | 2024

    Free access

    Multi-modal, multi-technique vehicle signal detection

    CREUSOT CLEMENT / THUREMELLA DIVYA / YU NA et al. | European Patent Office | 2024

    Free access