Unmanned aerial vehicles (UAVs) provide benefits through eco-friendliness, cost-effectiveness, and reduction of human risk. Deep reinforcement learning (DRL) is widely used for autonomous UAV navigation; however, current techniques often oversimplify the environment or impose movement restrictions. Additionally, most vision-based systems lack precise depth perception, while range finders provide a limited environmental overview, and LiDAR is energy-intensive. To address these challenges, this paper proposes VizNav, a modular DRL-based framework for autonomous UAV navigation in dynamic 3D environments without imposing conventional mobility constraints. VizNav incorporates the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm with Prioritized Experience Replay and Importance Sampling (PER) to improve performance in continuous action spaces and mitigate overestimations. Additionally, VizNav employs depth map images (DMIs) to enhance visual navigation by accurately estimating objects’ depth information, thereby improving obstacle avoidance. Empirical results show that VizNav, by leveraging TD3, improves navigation, and the inclusion of PER and DMI further boosts performance. Furthermore, the deployment of VizNav across various experimental settings confirms its flexibility and adaptability. The framework’s architecture separates the agent’s learning from the training process, facilitating integration with various DRL algorithms, simulation environments, and reward functions. This modularity creates a potential to influence RL simulation in various autonomous navigation systems, including robotics control and autonomous vehicles.


    Access

    Download


    Export, share and cite



    Title :

    VizNav: A Modular Off-Policy Deep Reinforcement Learning Framework for Vision-Based Autonomous UAV Navigation in 3D Dynamic Environments


    Contributors:


    Publication date :

    2024




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    Unknown




    PIC4rl-gym: a ROS2 Modular Framework for Robots Autonomous Navigation with Deep Reinforcement Learning

    Martini, Mauro / Eirale, Andrea / Cerrato, Simone et al. | BASE | 2023

    Free access

    Deep Reinforcement Learning for Autonomous Drone Navigation in Cluttered Environments

    Chandrashekhar, A / Rawate, Amit / Dhanamathi, A. et al. | IEEE | 2024


    Deep Reinforcement Learning for Autonomous Drone Navigation in Cluttered Environments

    Solaimalai, Gautam / Prakash, Kode Jaya / S, Sampath Kumar et al. | IEEE | 2024



    Vision-Based Deep Reinforcement Learning of UAV Autonomous Navigation Using Privileged Information

    Wang, Junqiao / Yu, Zhongliang / Zhou, Dong et al. | ArXiv | 2024

    Free access