We present an introspective failure prediction approach for autonomous vehicles. In autonomous driving, complex or unknown scenarios can cause a disengagement of the self-driving system. Disengagements can be triggered either by automatic safety measures or by human intervention. We propose to use recorded disengagement sequences from test drives as training data to learn to predict future failures. The system then learns introspectively from its own previous mistakes. In order to predict failures as early as possible, we propose a machine learning approach where sequences of sensor data are classified as either failure or success. The car itself is treated as a black box. Our method combines two sensor modalities that contain different types of information. An image-based model learns to detect generally challenging situations such as crowded intersections accurately multiple seconds in advance. A state data based model allows to detect fast changes immediately before a failure, such as sudden braking or swerving. The outcome of the individual models is fused by averaging the individual failure probabilities. We evaluate our approach on a data set provided by the BMW Group containing 14 hours of autonomous driving. The proposed late fusion approach allows for predicting failures at an accuracy of more than 85% seven seconds in advance, at a false positive rate of 20%. The proposed method outperforms state-of-the-art failure prediction by more than 15% while being a flexible framework that allows for straightforward addition of further sensor modalities.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    Introspective Failure Prediction for Autonomous Driving Using Late Fusion of State and Camera Information


    Beteiligte:


    Erscheinungsdatum :

    01.05.2022


    Format / Umfang :

    2568476 byte




    Medientyp :

    Aufsatz (Zeitschrift)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch



    INTROSPECTIVE BLACK BOX FAILURE PREDICTION FOR AUTONOMOUS DRIVING

    Kuhn, Christopher B. / Hofbauer, Markus / Petrovic, Goran et al. | British Library Conference Proceedings | 2020


    Introspective Black Box Failure Prediction for Autonomous Driving

    Kuhn, Christopher B. / Hofbauer, Markus / Petrovic, Goran et al. | IEEE | 2020


    Introspective autonomous vehicle operational management

    SVEGLIATO JUSTIN / WITWICKI STEFAN / WRAY KYLE HOLLINS et al. | Europäisches Patentamt | 2020

    Freier Zugriff

    Introspective Failure Prediction for Semantic Image Segmentation

    Kuhn, Christopher B. / Hofbauer, Markus / Lee, Sungkyu et al. | IEEE | 2020


    Introspective Autonomous Vehicle Operational Management

    SVEGLIATO JUSTIN / WITWICKI STEFAN / WRAY KYLE HOLLINS et al. | Europäisches Patentamt | 2020

    Freier Zugriff