To assure the successful operation of connected and automated vehicles, it is critical to detect and isolate anomalous and/or faulty information in a timely manner. To do so, anomaly detection techniques should be implemented in real-time where if the probability of anomalous information exceeds a certain threshold, the information is dealt with accordingly. Traditionally, the threshold for judging whether the data is anomalous is fixed and determined a priori. However, not only does this approach fail to account for the feedback obtained during a trip on the performance of the algorithms, but it also fails to respond to potential changes in rates of anomalies. Hence, it is important to develop an approach that can dynamically alter this threshold in response to exogenous factors to assure reliable and robust system operation. We develop a mathematical framework which utilizes a dynamic threshold for an anomaly classification algorithm in order to maximize the safety of a trip. Specifically, we develop and pair an anomaly classification algorithm based on convolutional neural networks (CNN), with a partially observable Markov decision process (POMDP) model. We solve the resulting POMDP model using the asynchronous advantage actor critic (A3C) deep reinforcement learning algorithm. The prescribed policy determines the anomaly classification threshold in real-time that maximizes the performance. Our numerical experiments show that the POMDP model outperforms state-of-the-art benchmarks, especially under more difficult to detect anomaly profiles.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    A Dynamic Deep Reinforcement Learning-Bayesian Framework for Anomaly Detection


    Contributors:

    Published in:

    Publication date :

    2022-12-01


    Size :

    1831154 byte




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English



    Bayesian Filtering for Dynamic Anomaly Detection and Tracking

    Forti, Nicola / Millefiori, Leonardo M. / Braca, Paolo et al. | IEEE | 2022


    Reinforcement Learning-based Anomaly Detection for PHM applications

    Khan, Samir / Yairi, Takehisa / Nakasuka, Shinichi et al. | IEEE | 2022


    Self-Supervised Deep Learning Framework for Anomaly Detection in Traffic Data

    Morris, Clint / Yang, Jidong J. / Chorzepa, Mi Geum et al. | ASCE | 2022


    Blockchain and Deep Learning-Based Decentralized Anomaly Detection Framework for VANET

    Jetani, Harshil / Patel, Janam / Mahida, Nikunjkumar et al. | IEEE | 2024


    A Deep Learning Anomaly Detection Framework for Satellite Telemetry with Fake Anomalies

    Yakun Wang / Jianglei Gong / Jie Zhang et al. | DOAJ | 2022

    Free access