In this paper, we consider the dynamic content caching issue in the cache-enabled Internet of Things (IoT) systems. For real-time applications in cache-enabled IoT systems, it is imperative to design dynamic content caching schemes to reduce the energy consumption of sensors and improve the freshness of information at users. We first design a dynamic content caching procedure for a cache-enabled IoT system with limited cache capacity and express the evolution of the Age of Information (AoI) at both the edge caching node and each user. Then, we formulate the dynamic content caching problem as a Markov Decision Process to minimize the expectation of a long-term accumulative cost, which jointly considers the average AoI of users and the energy consumption of sensors. To solve this problem, we propose an actor-critic based caching algorithm without prior knowledge of users’ content demands. The numerical results show that the proposed algorithm can achieve lower average AoI and energy consumption than other baselines.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Dynamic Content Caching Based on Actor-Critic Reinforcement Learning for IoT Systems


    Contributors:
    Lai, Lifeng (author) / Zheng, Fu-Chun (author) / Wen, Wanli (author) / Luo, Jingjing (author) / Li, Ge (author)


    Publication date :

    2022-09-01


    Size :

    832205 byte





    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English




    Actor-Critic Reinforcement Learning for Control With Stability Guarantee

    Han, M / Zhang, L / Wang, J et al. | BASE | 2020

    Free access

    Actor-Critic Traction Control Based on Reinforcement Learning with Open-Loop Training

    Funk Drechsler, Maikol / Fiorentin, Thiago Antonio / Göllinger, Harald | BASE | 2022

    Free access


    Improving Generalization of Reinforcement Learning with Minimax Distributional Soft Actor-Critic

    Ren, Yangang / Duan, Jingliang / Li, Shengbo Eben et al. | IEEE | 2020