Abstract This study develops an autonomous artificial intelligence (AI) agent to detect anomalies in traffic flow time series data, which can learn anomaly patterns from data without supervision, requiring no ground-truth labels for model training or knowledge of a threshold for anomaly definition. Specifically, our model is based on reinforcement learning, where an agent is built by a Long-Short-Term-Memory (LSTM) model and Q-learning algorithm to incorporate sequential information in time series data into policy optimization. The key contribution of our model is the development of a novel unsupervised reward learning algorithm that automatically learns the reward for an action taken by the agent based on the distribution of data, without requiring a manual specification of a reward function. To test the performance of our model, we conduct a comprehensive set of experimental study on both real-world data from Brisbane city, Australia, and synthetic data simulated according to the distribution of real-world data. We compare the performance of our model against three state-of-the-art models, and the experimental results show that our model outperforms the other models in different parameter settings, with around 90% precision, 80% recall, and 85% F1 score.

    Highlights Applies deep reinforcement learning for time series anomaly detection. Proposes an unsupervised reward learning algorithm that learns the reward function from data Does not require ground-truth labels or thresholds for determining anomalies


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    Autonomous anomaly detection on traffic flow time series with reinforcement learning


    Beteiligte:
    He, Dan (Autor:in) / Kim, Jiwon (Autor:in) / Shi, Hua (Autor:in) / Ruan, Boyu (Autor:in)


    Erscheinungsdatum :

    2023-03-04




    Medientyp :

    Aufsatz (Zeitschrift)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch




    Adaptable Anomaly Detection in Traffic Flow Time Series

    Alam, Md Rakibul / Gerostathopoulos, Ilias / Amini, Sasan et al. | IEEE | 2019


    Deep Reinforcement Learning for Autonomous Traffic Light Control

    Garg, Deepeka / Chli, Maria / Vogiatzis, George | IEEE | 2018



    Reinforcement Learning-based Anomaly Detection for PHM applications

    Khan, Samir / Yairi, Takehisa / Nakasuka, Shinichi et al. | IEEE | 2022


    Spacecraft Time-Series Online Anomaly Detection Using Deep Learning

    Baireddy, Sriram / Desai, Sundip R. / Foster, Richard H. et al. | IEEE | 2023