Integrated sensing and communication (ISAC) is considered a key technology supporting Beyond-5G/6G (B5G/6G) networks, which allows the spectrum resources to be used for both sensing and communication. In this paper, we investigate an unmanned aerial vehicle (UAV)-enabled ISAC scenario, where the UAV sends ISAC signals to communicate with multiple users (UEs) and senses potential targets simultaneously, and a reconfigurable intelligent surface (RIS) is deployed to enhance the communication performance. Aiming at maximizing the sum-rate throughput of the system, we formulate the joint optimization problem of the trajectory and the beamforming matrix of the UAV, the passive beamforming matrix of the RIS. Currently, many researchers are working on using deep reinforcement learning (DRL) to address such problems due to its non-convex nature; however, as the environment becomes increasingly complex, high-dimensional state space and action space lead to a decrease in the performance of DRL. To tackle this issue, we propose a novel hierarchical deep reinforcement learning (HDRL) framework to solve the optimization problem. Through decomposing the original problem into the trajectory optimization problem and the sum-rate throughput optimization problem, we adopt a hierarchical twin-delayed deep deterministic policy gradient (HTD3) structure to optimize them alternately. The experimental results demonstrate that the obtained system sum-rate throughputs of the proposed HDRL with an HTD3 structure are 33%, 50%, and 10% higher than those obtained by TD3, twin-TD3 (TTD3), and TD3 with hovering only (TD3HO), respectively.


    Zugriff

    Download


    Exportieren, teilen und zitieren



    Titel :

    A Hierarchical Deep Reinforcement Learning Approach for Throughput Maximization in Reconfigurable Intelligent Surface-Aided Unmanned Aerial Vehicle–Integrated Sensing and Communication Network


    Beteiligte:
    Haitao Chen (Autor:in) / Jiansong Miao (Autor:in) / Ruisong Wang (Autor:in) / Hao Li (Autor:in) / Xiaodan Zhang (Autor:in)


    Erscheinungsdatum :

    2024




    Medientyp :

    Aufsatz (Zeitschrift)


    Format :

    Elektronische Ressource


    Sprache :

    Unbekannt




    Cognitive Radio Network Throughput Maximization with Deep Reinforcement Learning

    Ong, Kevin Shen Hoong / Zhang, Yang / Niyato, Dusit | IEEE | 2019


    Unmanned Aerial Vehicle-Assisted Reconfigurable Intelligent Surface for Energy Efficient and Reliable Communication

    Kumaravelu, Vinoth Babu / Jadhav, Hindavi Kishor / B. S., Anjana et al. | Springer Verlag | 2022


    DYNAMIC CONTROL OF UNMANNED AERIAL VEHICLE USING RECONFIGURABLE INTELLIGENT SURFACE

    FRANCESCO DEVOTI / PLACIDO MURSIA / VINCENZO SCIANCALEPORE et al. | Europäisches Patentamt | 2022

    Freier Zugriff

    Reconfigurable unmanned aerial vehicle

    YANG PENGQIAN / NI JUN / JIN ZIHAN et al. | Europäisches Patentamt | 2023

    Freier Zugriff