To achieve persistent systems in the future, autonomous underwater vehicles (AUVs) will need to autonomously dock onto a charging station. Here, reinforcement learning strategies were applied for the first time to control the docking of an AUV onto a fixed platform in a simulation environment. Two reinforcement learning schemes were investigated: one with continuous state and action spaces, deep deterministic policy gradient (DDPG), and one with continuous state but discrete action spaces, deep Q network (DQN). For DQN, the discrete actions were selected as step changes in the control input signals. The performance of the reinforcement learning strategies was compared with classical and optimal control techniques. The control actions selected by DDPG suffer from chattering effects due to a hyperbolic tangent layer in the actor. Conversely, DQN presents the best compromise between short docking time and low control effort, whilst meeting the docking requirements. Whereas the reinforcement learning algorithms present a very high computational cost at training time, they are five orders of magnitude faster than optimal control at deployment time, thus enabling an on-line implementation. Therefore, reinforcement learning achieves a performance similar to optimal control at a much lower computational cost at deployment, whilst also presenting a more general framework.


    Access

    Download


    Export, share and cite



    Title :

    Docking Control of an Autonomous Underwater Vehicle Using Reinforcement Learning


    Contributors:
    Anderlini, E (author) / Parker, GG (author) / Thomas, G (author)

    Publication date :

    2019-09-01


    Remarks:

    Applied Sciences , 9 (17) , Article 3456. (2019)


    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English



    Classification :

    DDC:    629




    Autonomous Underwater Vehicle Docking

    Bellingham, James G. | Springer Verlag | 2016


    UNDERWATER DOCKING SYSTEM FOR AUTONOMOUS UNDERWATER VEHICLE

    SAKAUE HIROSHI / MUKAIDA MINEHIKO / OKAYA NORIYUKI et al. | European Patent Office | 2018

    Free access

    AUV (Autonomous Underwater Vehicle) underwater docking device

    SHUI / LIU YUN / MOU CHUNXIAO et al. | European Patent Office | 2024

    Free access

    Underwater docking system for autonomous underwater vehicle

    SAKAUE HIROSHI / MUKAIDA MINEHIKO / OKAYA NORIYUKI et al. | European Patent Office | 2019

    Free access