Solving the decision-making problem between pursuing the objective of covering distance and exploiting thermal updrafts is the central challenge in cross-country soaring flight. The need for trading short-term rewarding actions against actions that pay off in the long term makes for a hard-to-solve problem. Policies resulting from reinforcement learning offer the potential to handle long-term correlations between actions taken and rewards received. The paper presents a reinforcement learning setup, which results in a control strategy for the autonomous soaring sample application of GPS Triangle racing. First, we frame the problem in terms of a Markov decision process. In particular, we present a straightforward model for the three-degrees-of-freedom system dynamics of a glider aircraft that does not make any simplifying assumptions regarding the wind field or the relative aircraft velocity. The competition task is decomposed into subtasks, then. Stochastic gradient ascent solves the associated hierarchical reinforcement learning problem without the designer employing any further, potentially deficient heuristics. We present an implementation of the overall policy alongside an updraft estimator on embedded hardware aboard an unpiloted glider aircraft. Flight-test results validate the successful transfer of the hierarchical control policy trained in simulation to real-world autonomous cross-country soaring.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Hierarchical Reinforcement Learning Approach for Autonomous Cross-Country Soaring


    Contributors:

    Published in:

    Publication date :

    2022-10-13


    Size :

    13 pages




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English



    Hierarchical Reinforcement Learning Approach Towards Autonomous Cross-Country Soaring

    Notter, Stefan / Schimpf, Fabian / Fichter, Walter | AIAA | 2021


    HIERARCHICAL REINFORCEMENT LEARNING APPROACH TOWARDS AUTONOMOUS CROSS-COUTRY SOARING

    Notter, Stefan / Schimpf, Fabian / Fichter, Walter | TIBKAT | 2021



    Autonomous Soaring

    V. P. Lin | NTIS | 2007


    Autonomous Soaring

    Lin, Victor P. | NTRS | 2007