The identification of learning mechanisms for locomotion has been the subject of much research for some time but many challenges remain. Dynamic systems theory (DST) offers a novel approach to humanoid learning through environmental interaction. Reinforcement learning (RL) has offered a promising method to adaptively link the dynamic system to the environment it interacts with via a reward-based value system. In this paper, we propose a model that integrates the above perspectives and applies it to the case of a humanoid (NAO) robot learning to walk the ability of which emerges from its value-based interaction with the environment. In the model, a simplified central pattern generator (CPG) architecture inspired by neuroscientific research and DST is integrated with an actor-critic approach to RL (cpg-actor-critic). In the cpg-actor-critic architecture, least-square-temporal-difference based learning converges to the optimal solution quickly by using natural gradient learning and balancing exploration and exploitation. Futhermore, rather than using a traditional (designer-specified) reward it uses a dynamic value function as a stability indicator that adapts to the environment. The results obtained are analyzed using a novel DST-based embodied cognition approach. Learning to walk, from this perspective, is a process of integrating levels of sensorimotor activity and value.


    Zugriff

    Download


    Exportieren, teilen und zitieren



    Titel :

    Humanoids learning to walk : a natural CPG-actor-critic architecture


    Beteiligte:
    Li, Cai (Autor:in) / Lowe, Robert (Autor:in) / Ziemke, Tom (Autor:in)

    Erscheinungsdatum :

    2013-01-01


    Anmerkungen:

    PMID 23675345



    Medientyp :

    Aufsatz (Zeitschrift)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch



    Klassifikation :

    DDC:    629




    Actor-Critic Policy Learning in Cooperative Planning

    Redding, Joshua / Geramifard, Alborz / Choi, Han-Lim et al. | AIAA | 2010


    Intrinsically Motivated Actor-Critic for Robot Motor Learning

    Hafez, Muhammad Burhan / Universität Hamburg | TIBKAT | 2020


    Actor-Critic Reinforcement Learning for Control With Stability Guarantee

    Han, M / Zhang, L / Wang, J et al. | BASE | 2020

    Freier Zugriff

    Improving Generalization of Reinforcement Learning with Minimax Distributional Soft Actor-Critic

    Ren, Yangang / Duan, Jingliang / Li, Shengbo Eben et al. | IEEE | 2020