This paper aims to solve the path following problem for an underactuated unmanned-surface-vessel (USV) based on deep reinforcement learning (DRL). A smoothly-convergent DRL (SCDRL) method is proposed based on the deep Q network (DQN) and reinforcement learning. In this new method, an improved DQN structure was developed as a decision-making network to reduce the complexity of the control law for the path following of a three-degree of freedom USV model. An exploring function was proposed based on the adaptive gradient descent to extract the training knowledge for the DQN from the empirical data. In addition, a new reward function was designed to evaluate the output decisions of the DQN, and hence, to reinforce the decision-making network in controlling the USV path following. Numerical simulations were conducted to evaluate the performance of the proposed method. The analysis results demonstrate that the proposed SCDRL converges more smoothly than the traditional deep Q learning while the path following error of the SCDRL is comparable to existing methods. Thanks to good usability and generality of the proposed method for USV path following, it can be applied to practical applications.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    Path Following Optimization for an Underactuated USV Using Smoothly-Convergent Deep Reinforcement Learning


    Beteiligte:
    Zhao, Yujiao (Autor:in) / Qi, Xin (Autor:in) / Ma, Yong (Autor:in) / Li, Zhixiong (Autor:in) / Malekian, Reza (Autor:in) / Sotelo, Miguel Angel (Autor:in)


    Erscheinungsdatum :

    01.10.2021


    Format / Umfang :

    3303536 byte




    Medientyp :

    Aufsatz (Zeitschrift)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch



    Path-following of Underactuated Underwater Vehicles

    Do, Khac Duc / Pan, Jie | Springer Verlag | 2009


    Path-following of Underactuated Ships Using Polar Coordinates

    Do, Khac Duc / Pan, Jie | Springer Verlag | 2009


    On path following of underactuated surface ships

    Li, Ronghui / Qi, Yunpeng / Li, Tieshan et al. | IEEE | 2014


    Path-following of Underactuated Ships Using Serret–Frenet Coordinates

    Do, Khac Duc / Pan, Jie | Springer Verlag | 2009