To achieve persistent systems in the future, autonomous underwater vehicles (AUVs) will need to autonomously dock onto a charging station. Here, reinforcement learning strategies were applied for the first time to control the docking of an AUV onto a fixed platform in a simulation environment. Two reinforcement learning schemes were investigated: one with continuous state and action spaces, deep deterministic policy gradient (DDPG), and one with continuous state but discrete action spaces, deep Q network (DQN). For DQN, the discrete actions were selected as step changes in the control input signals. The performance of the reinforcement learning strategies was compared with classical and optimal control techniques. The control actions selected by DDPG suffer from chattering effects due to a hyperbolic tangent layer in the actor. Conversely, DQN presents the best compromise between short docking time and low control effort, whilst meeting the docking requirements. Whereas the reinforcement learning algorithms present a very high computational cost at training time, they are five orders of magnitude faster than optimal control at deployment time, thus enabling an on-line implementation. Therefore, reinforcement learning achieves a performance similar to optimal control at a much lower computational cost at deployment, whilst also presenting a more general framework.


    Zugriff

    Download


    Exportieren, teilen und zitieren



    Titel :

    Docking Control of an Autonomous Underwater Vehicle Using Reinforcement Learning


    Beteiligte:
    Anderlini, E (Autor:in) / Parker, GG (Autor:in) / Thomas, G (Autor:in)

    Erscheinungsdatum :

    01.09.2019


    Anmerkungen:

    Applied Sciences , 9 (17) , Article 3456. (2019)


    Medientyp :

    Aufsatz (Zeitschrift)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch



    Klassifikation :

    DDC:    629




    Autonomous Underwater Vehicle Docking

    Bellingham, James G. | Springer Verlag | 2016


    UNDERWATER DOCKING SYSTEM FOR AUTONOMOUS UNDERWATER VEHICLE

    SAKAUE HIROSHI / MUKAIDA MINEHIKO / OKAYA NORIYUKI et al. | Europäisches Patentamt | 2018

    Freier Zugriff

    AUV (Autonomous Underwater Vehicle) underwater docking device

    SHUI / LIU YUN / MOU CHUNXIAO et al. | Europäisches Patentamt | 2024

    Freier Zugriff

    Underwater docking system for autonomous underwater vehicle

    SAKAUE HIROSHI / MUKAIDA MINEHIKO / OKAYA NORIYUKI et al. | Europäisches Patentamt | 2019

    Freier Zugriff