Machine Learning (ML) components are widely adopted in autonomous vehicles to perform tasks such as perception and planning. Despite the multiple uses of machine learning components and their benefits, incorrect outputs from machine learning components can compromise the safety of the system. The limitations of the machine learning algorithms and their acceptable level of performance that results in a reasonable level of residual risk are considered as a part of ISO 21448, the safety of the intended functionality (SOTIF) standard. Currently, to measure the performance of machine learning models, statistical metrics such as accuracy, recall, precision, and F1-measure are often used depending on the nature of the data and task. While these metrics help in understanding which machine learning model is better and can be chosen as a part of the vehicle’s architecture, they do not provide much information regarding safety, in particular, SOTIF. There is a need for new metrics to better assess safety corresponding to these machine learning models. The new metrics need to focus more if an incorrect output from the model results in crashes and near crashes and aid in proposing design changes that help to reduce the residual risk of the vehicle. To achieve this goal, in this paper we discuss the limitation of current metrics with an example architecture that uses machine learning models and propose new scenario-based metrics that help in better analysis of machine learning models for SOTIF.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    Metrics for Machine Learning Models to Facilitate SOTIF Analysis in Autonomous Vehicles


    Weitere Titelangaben:

    Sae Technical Papers


    Beteiligte:

    Kongress:

    WCX SAE World Congress Experience ; 2023



    Erscheinungsdatum :

    2023-04-11




    Medientyp :

    Aufsatz (Konferenz)


    Format :

    Print


    Sprache :

    Englisch





    Runtime unknown unsafe scenarios identification for SOTIF of autonomous vehicles

    Wang, Cheng / Storms, Kai / Zhang, Ning et al. | Elsevier | 2023


    Das SOTIF-Vorgehensmodell

    Schnieder, Lars / Hosse, René S. | Springer Verlag | 2019


    Das SOTIF-Vorgehensmodell

    Schnieder, Lars / Hosse, René S. | Springer Verlag | 2020