Integrated sensing and communication (ISAC) is considered a key technology supporting Beyond-5G/6G (B5G/6G) networks, which allows the spectrum resources to be used for both sensing and communication. In this paper, we investigate an unmanned aerial vehicle (UAV)-enabled ISAC scenario, where the UAV sends ISAC signals to communicate with multiple users (UEs) and senses potential targets simultaneously, and a reconfigurable intelligent surface (RIS) is deployed to enhance the communication performance. Aiming at maximizing the sum-rate throughput of the system, we formulate the joint optimization problem of the trajectory and the beamforming matrix of the UAV, the passive beamforming matrix of the RIS. Currently, many researchers are working on using deep reinforcement learning (DRL) to address such problems due to its non-convex nature; however, as the environment becomes increasingly complex, high-dimensional state space and action space lead to a decrease in the performance of DRL. To tackle this issue, we propose a novel hierarchical deep reinforcement learning (HDRL) framework to solve the optimization problem. Through decomposing the original problem into the trajectory optimization problem and the sum-rate throughput optimization problem, we adopt a hierarchical twin-delayed deep deterministic policy gradient (HTD3) structure to optimize them alternately. The experimental results demonstrate that the obtained system sum-rate throughputs of the proposed HDRL with an HTD3 structure are 33%, 50%, and 10% higher than those obtained by TD3, twin-TD3 (TTD3), and TD3 with hovering only (TD3HO), respectively.


    Access

    Download


    Export, share and cite



    Title :

    A Hierarchical Deep Reinforcement Learning Approach for Throughput Maximization in Reconfigurable Intelligent Surface-Aided Unmanned Aerial Vehicle–Integrated Sensing and Communication Network


    Contributors:
    Haitao Chen (author) / Jiansong Miao (author) / Ruisong Wang (author) / Hao Li (author) / Xiaodan Zhang (author)


    Publication date :

    2024




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    Unknown




    Cognitive Radio Network Throughput Maximization with Deep Reinforcement Learning

    Ong, Kevin Shen Hoong / Zhang, Yang / Niyato, Dusit | IEEE | 2019


    Unmanned Aerial Vehicle-Assisted Reconfigurable Intelligent Surface for Energy Efficient and Reliable Communication

    Kumaravelu, Vinoth Babu / Jadhav, Hindavi Kishor / B. S., Anjana et al. | Springer Verlag | 2022


    DYNAMIC CONTROL OF UNMANNED AERIAL VEHICLE USING RECONFIGURABLE INTELLIGENT SURFACE

    FRANCESCO DEVOTI / PLACIDO MURSIA / VINCENZO SCIANCALEPORE et al. | European Patent Office | 2022

    Free access


    Reconfigurable unmanned aerial vehicle

    YANG PENGQIAN / NI JUN / JIN ZIHAN et al. | European Patent Office | 2023

    Free access