Locomotion control of legged robots is a challenging problem. Recently, reinforcement learning has been applied to legged locomotion and made a great success. However, the reward signal design remains a challenging problem to produce a humanlike motion such as walking and running. Although imitation learning provides a way to mimic the behavior of humans or animals, the obtained motion may be restricted due to the over-constrained property of this method. Here we propose a novel and simple way to generate humanlike behavior by using feedforward enhanced reinforcement learning (FERL). In FERL, the control action is composed of a feedforward part and a feedback part, where the feedforward part is a periodic time-dependent signal generated by a state machine and the feedback part is a state-dependent signal obtained by a neural network. By using FERL with a simple feedforward of two feet stepping up and down alternately, we achieve humanlike walking and running for a simulated biped robot, Ranger Max. Comparison results show that the feedforward is key to generating humanlike behavior, while the policy trained with no feedforward only results in some strange gaits. FERL may also be extended to other legged robots to generate various locomotion styles, which provides a competitive alternative for imitation learning.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Realizing Human-like Walking and Running with Feedforward Enhanced Reinforcement Learning


    Additional title:

    Lect.Notes Computer


    Contributors:
    Yang, Huayong (editor) / Liu, Honghai (editor) / Zou, Jun (editor) / Yin, Zhouping (editor) / Liu, Lianqing (editor) / Yang, Geng (editor) / Ouyang, Xiaoping (editor) / Wang, Zhiyong (editor) / Ye, Linqi (author) / Wang, Xueqian (author)

    Conference:

    International Conference on Intelligent Robotics and Applications ; 2023 ; Hangzhou, China July 05, 2023 - July 07, 2023



    Publication date :

    2023-10-16


    Size :

    13 pages




    Type of media :

    Article/Chapter (Book)


    Type of material :

    Electronic Resource


    Language :

    English





    Reinforcement Learning Using Feedforward Neural Network with Memory Mechanism

    Ozawa, S. / Shiraga, N. | British Library Online Contents | 2003


    Exoskeletons for running and walking

    HERR HUGH M / WALSH CONOR / PALUSKA DANIEL JOSEPH et al. | European Patent Office | 2019

    Free access

    A Parallel Feedforward Compensator Realizing ASPRness for Plants with Structured Uncertainty

    Deng, M. / Mizumoto, I. / Iwai, Z. | British Library Online Contents | 1997


    Elevator running along walking stairs

    ZHENG YONGRU | European Patent Office | 2016

    Free access