Abstract Detecting anomalies in telemetry data captured on-board a spacecraft is critical to ensure its safe operation. Although there exist various techniques for automatically detecting point, contextual, and collective anomalies from time-series data, quantifying their performance remains under-researched. In this paper, we thoroughly validate our approach for the task of anomalous event detection that is built upon a two-stage technique, in which the telemetry signal is predicted using a long short-term memory network based on the historical data, and then the prediction is compared with the actual (captured) data. If the difference between those two is sufficiently large, we can infer that an anomalous event has happened. To evaluate the capabilities of such detection techniques over the simulated and benchmark time-series data, we investigate a set of commonly used metrics obtained for a range of anomaly detection approaches, and present their shortcomings, especially related to their inability of capturing the temporal aspects of the detectors. We tackle this issue by introducing new quality metrics which enable us to objectively verify if the detectors can timely spot the anomalies in sequential data. The experimental study showed that inferring the conclusions based on a subset of metrics can lead to biased observations, as the best algorithms determined based on the overlap metrics, including the Dice coefficient, do not necessarily correspond to the algorithms that offer the fastest detection. Finally, we discuss the Antelope Toolbox—our software tool for simulating nominal telemetry data of given characteristics, alongside well-defined anomalous events, and to perform the quantitative and qualitative analysis of anomaly detection algorithms over such simulated events.

    Highlights We investigate the quality metrics to verify anomaly detection in telemetry data. We introduce new measures that capture temporal aspects of anomaly detectors. We present an end-to-end machine learning pipeline for anomaly detection. We objectively assess 15 anomaly detection algorithms using all metrics. Exploiting only a subset of all metrics can easily lead to biased conclusions.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    Evaluating algorithms for anomaly detection in satellite telemetry data


    Beteiligte:

    Erschienen in:

    Acta Astronautica ; 198 ; 689-701


    Erscheinungsdatum :

    2022-06-20


    Format / Umfang :

    13 pages




    Medientyp :

    Aufsatz (Zeitschrift)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch




    EVALUATING ANOMALY DETECTION IN SATELLITE TELEMETRY DATA

    Nalepa, Jakub / Benecki, Pawel / Andrzejewski, Jacek et al. | TIBKAT | 2022


    Numerical Analysis of Automated Anomaly Detection Algorithms for Satellite Telemetry

    Schlag, Leonard / OMeara, Corey / Wickler, Martin | AIAA | 2018


    Supporting Anomaly Detection from Satellite Telemetry Data by Regression Trees

    Nakatsugawa, M. / Yairi, T. / Isihama, N. et al. | British Library Conference Proceedings | 2004


    A hybrid data-driven framework for satellite telemetry data anomaly detection

    Xu, Zhaoping / Cheng, Zhijun / Guo, Bo | Elsevier | 2023


    Numerical Analysis of Automated Anomaly Detection Algorithms for Satellite Telemetry (AIAA 2018-2534)

    Schlag, Leonard / OMeara, Corey / Wickler, Martin | British Library Conference Proceedings | 2018