Autonomous exploration is an essential task for unmanned aerial vehicles (UAVs) operating in unknown environments. Often, UAVs on these missions must first build a map of the environment via pure exploration and subsequently use (i.e. exploit) the generated map for downstream navigation tasks. Accomplishing these navigation tasks in two separate steps is not always possible and can potentially be disadvantageous for UAVs deployed in outdoor and dynamically changing environments. Current exploration approaches typically use a priori human-generated maps or heuristics such as frontier-based exploration. Other approaches use learning but focus only on learning policies for specific tasks and use sample inefficient random exploration or make impractical assumptions about full map availability. In this paper, we develop an adaptive exploration approach that allows for a trade off between exploration and exploitation in a single step using Deep Reinforcement Learning (DRL). We specifically focus on UAVs searching for areas of interest (AoIs) in an unknown environment. The proposed approach uses a map segmentation technique to decompose the environment map into smaller, tractable maps. DDQN and A2C algorithms are extended with a stack of LSTM layers and trained to generate optimal policies for the exploration and exploitation tasks, respectively. Then, an information gain function is repeatedly computed to determine the optimal trade-off between them. We tested our approach in 3 different tasks against 4 baselines. The results demonstrate that our proposed approach is capable of navigating through randomly generated environments and covering more AoI in less time compared to the baselines.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Deep Reinforcement Learning for Adaptive Exploration of Unknown Environments


    Contributors:


    Publication date :

    2021-06-15


    Size :

    2829396 byte





    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English



    Exploration of Unknown Environment using Deep Reinforcement Learning

    Ali, Asad / Gul, Sarah / Mahmood, Tallat et al. | IEEE | 2023



    A Reinforcement Learning Framework for Space Missions in Unknown Environments

    Tavallali, Peyman / Karumanchi, Sisir / Bowkett, Joseph et al. | IEEE | 2020


    A Reinforcement Learning Framework for Space Missions in Unknown Environments

    Kennedy, Brett / Bowkett, Joseph / Karumanchi, Sisir et al. | NTRS | 2020


    Navigation of Autonomous Vehicles in Unknown Environments Using Reinforcement Learning

    Martynez-Maryn, T. / Rodryguez, R. / Institute of Electrical and Electronics Engineers | British Library Conference Proceedings | 2007