When exploring an unknown environment, a mobile robot must decide where to observe next. It must do this whilst minimising the risk of failure, by only exploring areas that it expects to be safe. In this context, safety refers to the robot remaining in regions where critical environment features (e.g. terrain steepness, radiation levels) are within ranges the robot is able to tolerate. More specifically, we consider a setting where a robot explores an environment modelled with a Markov decision process, subject to bounds on the values of one or more environment features which can only be sensed at runtime. We use a Gaussian process to predict the value of the environment feature in unvisited regions, and propose an estimated Markov decision process, a model that integrates the Gaussian process predictions with the environment model transition probabilities. Building on this model, we propose an exploration algorithm that, contrary to previous approaches, considers probabilistic transitions and explicitly reasons about the uncertainty over the Gaussian process predictions. Furthermore, our approach increases the speed of exploration by selecting locations to visit further away from the currently explored area. We evaluate our approach on a real-world gamma radiation dataset, tackling the challenge of a nuclear material inspection robot exploring an a priori unknown area.


    Access

    Download


    Export, share and cite



    Title :

    Markov decision processes with unknown state feature values for safe exploration using Gaussian processes


    Contributors:
    Budd, M (author) / Lacerda, B (author) / Duckworth, P (author) / West, A (author) / Lennox, B (author) / Hawes, N (author)

    Publication date :

    2020-10-20



    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English


    Classification :

    DDC:    629




    Human Intent Prediction Using Markov Decision Processes

    McGhan, Catharine / Nasir, Ali / Atkins, Ella | AIAA | 2012


    Planning using hierarchical constrained Markov decision processes

    Feyzabadi, S. | British Library Online Contents | 2017


    Human Intent Prediction Using Markov Decision Processes

    McGhan, Catharine L. R. / Nasir, Ali / Atkins, Ella M. | AIAA | 2015


    Reconstruction Error Estimation of Gaussian Markov Processes with Jitter

    Espinoza-Bautista, Jose Rodrigo / Rodriguez-Saldana, Daniel / Kazakov, Vladimir | IEEE | 2015