Machine Learning (ML) components are widely adopted in autonomous vehicles to perform tasks such as perception and planning. Despite the multiple uses of machine learning components and their benefits, incorrect outputs from machine learning components can compromise the safety of the system. The limitations of the machine learning algorithms and their acceptable level of performance that results in a reasonable level of residual risk are considered as a part of ISO 21448, the safety of the intended functionality (SOTIF) standard. Currently, to measure the performance of machine learning models, statistical metrics such as accuracy, recall, precision, and F1-measure are often used depending on the nature of the data and task. While these metrics help in understanding which machine learning model is better and can be chosen as a part of the vehicle’s architecture, they do not provide much information regarding safety, in particular, SOTIF. There is a need for new metrics to better assess safety corresponding to these machine learning models. The new metrics need to focus more if an incorrect output from the model results in crashes and near crashes and aid in proposing design changes that help to reduce the residual risk of the vehicle. To achieve this goal, in this paper we discuss the limitation of current metrics with an example architecture that uses machine learning models and propose new scenario-based metrics that help in better analysis of machine learning models for SOTIF.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Metrics for Machine Learning Models to Facilitate SOTIF Analysis in Autonomous Vehicles


    Additional title:

    Sae Technical Papers


    Contributors:

    Conference:

    WCX SAE World Congress Experience ; 2023



    Publication date :

    2023-04-11




    Type of media :

    Conference paper


    Type of material :

    Print


    Language :

    English




    Metrics for Machine Learning Models to Facilitate SOTIF Analysis in Autonomous Vehicles

    Madala, Kaushik / Avalos Gonzalez, Carlos | British Library Conference Proceedings | 2023



    Das SOTIF-Vorgehensmodell

    Schnieder, Lars / Hosse, René S. | Springer Verlag | 2019



    Das SOTIF-Vorgehensmodell

    Schnieder, Lars / Hosse, René S. | Springer Verlag | 2020