Semantic segmentation of images enables pixel-wise scene understanding which in turn is a critical component for tasks such as autonomous driving. While recent implementations of semantic image segmentation have achieved remarkable accuracy, misclassifications remain inevitable. For safety-critical tasks such as free-space computing, it is desirable to know when and where the segmentation will fail. We propose using the concept of introspection to predict the failures of a given semantic segmentation model. A separate introspective model is trained to predict the errors of a given model. This is accomplished by training the given model with the errors made on a set of previous inputs. By using the same architecture for the introspective model as for the semantic segmentation, the proposed model learns to predict pixel-wise failure probabilities. This allows to predict both when and where the semantic segmentation will fail. Sharing the feature encoder with the inspected model reduces training and inference time while improving performance. We evaluate our approach on the large-scale A2D2 driving data set. In a precision-recall analysis, the proposed method outperforms two state-of-the-art uncertainty estimation methods by 3.2% and 6.7% while requiring significantly less resources during inference. Additionally, combining introspection with a state-of-the-art method further increases the performance by up to 3.7%.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    Introspective Failure Prediction for Semantic Image Segmentation


    Beteiligte:


    Erscheinungsdatum :

    2020-09-20


    Format / Umfang :

    3122208 byte




    Medientyp :

    Aufsatz (Konferenz)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch



    Introspective Black Box Failure Prediction for Autonomous Driving

    Kuhn, Christopher B. / Hofbauer, Markus / Petrovic, Goran et al. | IEEE | 2020


    Introspective autonomous vehicle operational management

    SVEGLIATO JUSTIN / WITWICKI STEFAN / WRAY KYLE HOLLINS et al. | Europäisches Patentamt | 2020

    Freier Zugriff

    Introspective Failure Prediction for Autonomous Driving Using Late Fusion of State and Camera Information

    Kuhn, Christopher B. / Hofbauer, Markus / Petrovic, Goran et al. | IEEE | 2022


    Introspective Autonomous Vehicle Operational Management

    SVEGLIATO JUSTIN / WITWICKI STEFAN / WRAY KYLE HOLLINS et al. | Europäisches Patentamt | 2020

    Freier Zugriff