This master thesis focuses on developing a Reinforcement Learning (RL) controller to perform hydrobatic maneuvers on an Autonomous Underwater Vehicle (AUV) successfully. This work also aims to analyze the robustness of the RL controller, as well as provide a comparison between RL algorithms and Proportional Integral Derivative (PID) control. Training of the algorithms is initially conducted in a Numpy simulation in Python. We show how to model the Equations of Motion (EOM) of the AUV and how to use it to train the RL controllers. We use the stablebaselines3 RL framework and create a training environment with the OpenAI gym. The Twin-Delay Deep Deterministic Policy Gradient (TD3) algorithm offers good performance in the simulation. The following maneuvers are studied: trim control, waypoint following, and an inverted pendulum. We test the maneuvers both in the Numpy simulation and Stonefish simulator. Also, we test the robustness of the RL trim controller by simulating noise in the state feedback. Lastly, we run the RL trim controller on a real AUV hardware called SAM. We show that the RL algorithm trained in the Numpy simulator can achieve similar performance to the PID controller in the Stonefish simulator. We generate a policy that can perform the trim control and the Inverted Pendulum maneuver in the Numpy simulation. We show that we can generate a robust policy that executes other types of maneuvers by providing a parameterized cost function to the RL algorithm. We discuss the results of every maneuver we perform with the SAM AUV and provide a discussion about the advantages and disadvantages of this control method applied to underwater robotics. We conclude that RL can be used to create policies that perform hydrobatic maneuvers. This data-driven approach can be applied in the future to more complex problems in underwater robotics. ; Denna masteruppsats fokuserar på att utveckla en Reinforcement Learning (RL) kontroller för att framgångsrikt utföra hydrobatiska manövrar på ett autonomt undervattensfordon ...


    Access

    Download


    Export, share and cite



    Title :

    Reinforcement Learning for Hydrobatic AUVs ; Reinforcement learning för Hydrobatiska AUV


    Contributors:

    Publication date :

    2022-01-01


    Type of media :

    Theses


    Type of material :

    Electronic Resource


    Language :

    English



    Classification :

    DDC:    629



    Deep Reinforcement Learning

    Huang, Xiaowei / Jin, Gaojie / Ruan, Wenjie | Springer Verlag | 2012


    Malthusian Reinforcement Learning

    Leibo, JZ / Perolat, J / Hughes, E et al. | BASE | 2019

    Free access

    Minimax Reinforcement Learning

    Chakravorty, Suman / Hyland, David | AIAA | 2003


    SAFE AND EFFICIENT REINFORCEMENT LEARNING ; Säker och effektiv reinforcement learning

    Magnusson, Björn / Forslund, Måns | BASE | 2019

    Free access