This study investigated the navigation problem for cellular-connected unmanned aerial vehicles (UAVs), particularly in highly dynamic urban environments. To address this problem, the UAV is required not only to evade high-speed obstacles in the airspace but also to avoid the coverage holes of cellular base stations (BS). Moreover, the UAV needs to reach the destination to complete the navigation task. Hence, it is imperative to design the trade-off in action selections between collision evasion and destination-approaching scenarios, while also considering the expected communication outage duration as a crucial reference. To overcome this multiobjective optimization challenge, we propose a deep reinforcement learning (DRL)-based algorithm aimed at enabling the UAV to acquire an optimal decision-making policy. Specifically, we formulated the navigation problem as a Markov decision process (MDP) and developed a layered recurrent soft actor–critic (RSAC)-based DRL framework, stimulating the UAV to resolve two fundamental subtasks of UAV navigation. Furthermore, we develop a multilayer perception (MLP)-based integrated evaluation network to select a particular action from the two subsolutions, satisfying the demands for the entire navigation problem. The layered architecture simplifies the navigation problem, thereby enhancing the convergence speed of the proposed algorithm. Numerical results indicate that the layered-RSAC-based UAV can autonomously perform scheduled navigation tasks in our designed simulated urban environments with superior effectiveness.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Autonomous Navigation for Cellular-Connected UAV in Highly Dynamic Environments: A Deep Reinforcement Learning Approach


    Additional title:

    J. Aerosp. Eng.


    Contributors:
    Wu, Di (author) / Shi, Zhiyi (author) / Zhang, Yibo (author) / Huang, Mengxing (author)

    Published in:

    Publication date :

    2024-09-01




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English




    Deep Reinforcement Learning for Autonomous Drone Navigation in Cluttered Environments

    Chandrashekhar, A / Rawate, Amit / Dhanamathi, A. et al. | IEEE | 2024


    Deep Reinforcement Learning for Autonomous Drone Navigation in Cluttered Environments

    Solaimalai, Gautam / Prakash, Kode Jaya / S, Sampath Kumar et al. | IEEE | 2024


    Autonomous vehicle navigation with deep reinforcement learning

    Cabañeros López, Àlex | BASE | 2019

    Free access