Locomotion control of legged robots is a challenging problem. Recently, reinforcement learning has been applied to legged locomotion and made a great success. However, the reward signal design remains a challenging problem to produce a humanlike motion such as walking and running. Although imitation learning provides a way to mimic the behavior of humans or animals, the obtained motion may be restricted due to the over-constrained property of this method. Here we propose a novel and simple way to generate humanlike behavior by using feedforward enhanced reinforcement learning (FERL). In FERL, the control action is composed of a feedforward part and a feedback part, where the feedforward part is a periodic time-dependent signal generated by a state machine and the feedback part is a state-dependent signal obtained by a neural network. By using FERL with a simple feedforward of two feet stepping up and down alternately, we achieve humanlike walking and running for a simulated biped robot, Ranger Max. Comparison results show that the feedforward is key to generating humanlike behavior, while the policy trained with no feedforward only results in some strange gaits. FERL may also be extended to other legged robots to generate various locomotion styles, which provides a competitive alternative for imitation learning.
Realizing Human-like Walking and Running with Feedforward Enhanced Reinforcement Learning
Lect.Notes Computer
International Conference on Intelligent Robotics and Applications ; 2023 ; Hangzhou, China July 05, 2023 - July 07, 2023
2023-10-16
13 pages
Article/Chapter (Book)
Electronic Resource
English
Reinforcement Learning , Biped Robot , Locomotion Control Computer Science , Artificial Intelligence , Software Engineering/Programming and Operating Systems , Computer Applications , User Interfaces and Human Computer Interaction , Computer Communication Networks , Special Purpose and Application-Based Systems
Reinforcement Learning Using Feedforward Neural Network with Memory Mechanism
British Library Online Contents | 2003
|A Parallel Feedforward Compensator Realizing ASPRness for Plants with Structured Uncertainty
British Library Online Contents | 1997
|