Locomotion control of legged robots is a challenging problem. Recently, reinforcement learning has been applied to legged locomotion and made a great success. However, the reward signal design remains a challenging problem to produce a humanlike motion such as walking and running. Although imitation learning provides a way to mimic the behavior of humans or animals, the obtained motion may be restricted due to the over-constrained property of this method. Here we propose a novel and simple way to generate humanlike behavior by using feedforward enhanced reinforcement learning (FERL). In FERL, the control action is composed of a feedforward part and a feedback part, where the feedforward part is a periodic time-dependent signal generated by a state machine and the feedback part is a state-dependent signal obtained by a neural network. By using FERL with a simple feedforward of two feet stepping up and down alternately, we achieve humanlike walking and running for a simulated biped robot, Ranger Max. Comparison results show that the feedforward is key to generating humanlike behavior, while the policy trained with no feedforward only results in some strange gaits. FERL may also be extended to other legged robots to generate various locomotion styles, which provides a competitive alternative for imitation learning.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    Realizing Human-like Walking and Running with Feedforward Enhanced Reinforcement Learning


    Weitere Titelangaben:

    Lect.Notes Computer


    Beteiligte:
    Yang, Huayong (Herausgeber:in) / Liu, Honghai (Herausgeber:in) / Zou, Jun (Herausgeber:in) / Yin, Zhouping (Herausgeber:in) / Liu, Lianqing (Herausgeber:in) / Yang, Geng (Herausgeber:in) / Ouyang, Xiaoping (Herausgeber:in) / Wang, Zhiyong (Herausgeber:in) / Ye, Linqi (Autor:in) / Wang, Xueqian (Autor:in)

    Kongress:

    International Conference on Intelligent Robotics and Applications ; 2023 ; Hangzhou, China July 05, 2023 - July 07, 2023



    Erscheinungsdatum :

    16.10.2023


    Format / Umfang :

    13 pages




    Medientyp :

    Aufsatz/Kapitel (Buch)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch





    Reinforcement Learning Using Feedforward Neural Network with Memory Mechanism

    Ozawa, S. / Shiraga, N. | British Library Online Contents | 2003


    A Parallel Feedforward Compensator Realizing ASPRness for Plants with Structured Uncertainty

    Deng, M. / Mizumoto, I. / Iwai, Z. | British Library Online Contents | 1997


    Exoskeletons for running and walking

    HERR HUGH M / WALSH CONOR / PALUSKA DANIEL JOSEPH et al. | Europäisches Patentamt | 2019

    Freier Zugriff

    Elevator running along walking stairs

    ZHENG YONGRU | Europäisches Patentamt | 2016

    Freier Zugriff