Methods and systems of estimating an accuracy of a neural network on out-of-distribution data. In-distribution accuracies of a plurality of machine learning models trained with in-distribution data are determined. The plurality of machine learning models includes a first model, and a remainder of models. In-distribution agreement is determined between (i) an output of the first machine learning model executed with an in-distribution dataset and (ii) outputs of a remainder of the plurality of machine learning models executed with the in-distribution dataset. The machine learning models are also executed with an unlabeled out-of-distribution dataset, and an out-of-distribution agreement is determined. The in-distribution agreement is compared with the out-of-distribution agreement. Based on a result of the comparison being within a threshold, an accuracy of the first machine learning model on the unlabeled out-of-distribution dataset is estimated based on (i) the in-distribution accuracies, (ii) the in-distribution agreement, and (iii) the out-of-distribution agreement.


    Zugriff

    Download


    Exportieren, teilen und zitieren



    Titel :

    PERFORMANCE OF NEURAL NETWORKS UNDER DISTRIBUTION SHIFT


    Beteiligte:
    JIANG YIDING (Autor:in) / BAEK CHRISTINA (Autor:in) / KOLTER JEREMY (Autor:in) / RAGHUNATHAN ADITI (Autor:in) / SEMEDO JOÃO D (Autor:in) / CABRITA CONDESSA FILIPE J (Autor:in) / LIN WAN-YI (Autor:in)

    Erscheinungsdatum :

    2023-12-21


    Medientyp :

    Patent


    Format :

    Elektronische Ressource


    Sprache :

    Englisch


    Klassifikation :

    IPC:    B60W CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION , Gemeinsame Steuerung oder Regelung von Fahrzeug-Unteraggregaten verschiedenen Typs oder verschiedener Funktion / G06V



    Unsupervised Anomaly Detection Approach for Shift Quality Assessment Using Deep Neural Networks

    Oh, Geesung / Park, Joonghoo / Hwang, Kyunghun et al. | IEEE | 2022




    High Performance Simulation of Neural Networks

    Rademacher, T. J. / Lumpp, J. E. / IEEE; Aerospace and Electronics Systems Society | British Library Conference Proceedings | 1997


    Distribution Functions as Attractor for Recurrent Neural Networks

    Gimenez, V. / Aslanyan, L. / Castellanos, J. et al. | British Library Online Contents | 2001