One of the important certification objectives for airborne software is demonstrating the absence of unintended behavior. In current software development processes, unintended behavior is associated with some identifiable structural feature, such as specific lines of code or a model element. However, in learning-enabled systems (like neural networks or other machine learning approaches), unintended behavior emerges from the data used to train the system. New inputs not encountered during training may result in novel activations in a neural network, leading to unexpected (and potentially dangerous) outputs. In this paper we will first review the rationale and methods for detecting unintended behavior in current airborne software systems, including the use of model based development techniques and formal methods for software verification. Then we will consider the challenges posed by learning-enabled components (LECs) and examine new techniques that are being developed to address these challenges, as well as how these techniques may shape new certification guidance.
Unintended Behavior in Learning-Enabled Systems: Detecting the Unknown Unknowns
2021-10-03
1212437 byte
Aufsatz (Konferenz)
Elektronische Ressource
Englisch
Uncovering unknown unknowns: Towards a Baconian approach to management decision-making
Online Contents | 2014
|A VALIDATION METHODOLOGY FOR THE MINIMIZATION OF UNKNOWN UNKNOWNS IN AUTONOMOUS VEHICLE SYSTEMS
British Library Conference Proceedings | 2020
|OPERATIONS - 'Vigilant Shield '07' games helps prepare troops for the 'unknown unknowns'
Online Contents | 2007
Apparatus for detecting vehicle sudden unintended acceleration
Europäisches Patentamt | 2024
|