Methods and systems of estimating an accuracy of a neural network on out-of-distribution data. In-distribution accuracies of a plurality of machine learning models trained with in-distribution data are determined. The plurality of machine learning models includes a first model, and a remainder of models. In-distribution agreement is determined between (i) an output of the first machine learning model executed with an in-distribution dataset and (ii) outputs of a remainder of the plurality of machine learning models executed with the in-distribution dataset. The machine learning models are also executed with an unlabeled out-of-distribution dataset, and an out-of-distribution agreement is determined. The in-distribution agreement is compared with the out-of-distribution agreement. Based on a result of the comparison being within a threshold, an accuracy of the first machine learning model on the unlabeled out-of-distribution dataset is estimated based on (i) the in-distribution accuracies, (ii) the in-distribution agreement, and (iii) the out-of-distribution agreement.
PERFORMANCE OF NEURAL NETWORKS UNDER DISTRIBUTION SHIFT
2023-12-21
Patent
Elektronische Ressource
Englisch
Modeling freight distribution using artificial neural networks
Elsevier | 2003
|Modeling freight distribution using artificial neural networks
Online Contents | 2004
|High Performance Simulation of Neural Networks
British Library Conference Proceedings | 1997
|Distribution Functions as Attractor for Recurrent Neural Networks
British Library Online Contents | 2001
|