Reinforcement learning (RL) has proven to be a powerful paradigm for deriving complex behaviors from simple reward signals in a wide range of environments. When applying RL to continuous control agents in simulated physics environments, the body is usually considered to be part of the environment. However, during evolution the physical body of biological organisms and their controlling brains are co-evolved, thus exploring a much larger space of actuator/controller configurations. Put differently, the intelligence does not reside only in the agent’s mind, but also in the design of their body. We propose a method for uncovering strong agents, consisting of a good combination of a body and policy, based on combining RL with an evolutionary procedure. Given the resulting agent, we also propose an approach for identifying the body changes that contributed the most to the agent performance. We use the Shapley value from cooperative game theory to find the fair contribution of individual components, taking into account synergies between components. We evaluate our methods in an environment similar to the the recently proposed Robo-Sumo task, where agents in a software physics simulator compete in tipping over their opponent or pushing them out of the arena. Our results show that the proposed methods are indeed capable of generating strong agents, significantly outperforming baselines that focus on optimizing the agent policy alone.


    Access

    Download


    Export, share and cite



    Title :

    The Body is Not a Given: Joint Agent Policy Learning and Morphology Evolution


    Contributors:
    Banarse, D (author) / Bachrach, Y (author) / Liu, S (author) / Lever, G (author) / Heess, N (author) / Fernando, C (author) / Kohli, P (author) / Graepel, T (author) / Elkind, Edith / Veloso, Manuela

    Publication date :

    2019-05-17


    Remarks:

    In: Elkind, Edith and Veloso, Manuela, (eds.) Proceedings of the 18th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2019). (pp. pp. 1134-1142). IFAAMAS (International Foundation of Autonomous Agents and MultiAgent Systems): Montreal, Canada. (2019)


    Type of media :

    Paper


    Type of material :

    Electronic Resource


    Language :

    English



    Classification :

    DDC:    006 / 629



    Transport policy in Australia—Evolution, learning and policy transfer

    Bray, David J. / Taylor, Michael A.P. / Scrafton, Derek | Elsevier | 2010



    General guidelines given for tubular joint design (1)

    Bynum, D. / George, H.H. | Tema Archive | 1979


    Anti-drone Policy Learning Based on Self-attention Multi-agent Deterministic Policy Gradient

    Jiang, Tianrui / Zhuang, Dongye / Xie, Haibin | Springer Verlag | 2022


    JOINT STRUCTURE, JOINT BODY, AND STRUCTURAL BODY

    TAGUCHI MAKOTO / SUGIMOTO SUNAO / ISHIKAWA TAKESHI et al. | European Patent Office | 2023

    Free access