Reinforcement learning (RL) has proven to be a powerful paradigm for deriving complex behaviors from simple reward signals in a wide range of environments. When applying RL to continuous control agents in simulated physics environments, the body is usually considered to be part of the environment. However, during evolution the physical body of biological organisms and their controlling brains are co-evolved, thus exploring a much larger space of actuator/controller configurations. Put differently, the intelligence does not reside only in the agent’s mind, but also in the design of their body. We propose a method for uncovering strong agents, consisting of a good combination of a body and policy, based on combining RL with an evolutionary procedure. Given the resulting agent, we also propose an approach for identifying the body changes that contributed the most to the agent performance. We use the Shapley value from cooperative game theory to find the fair contribution of individual components, taking into account synergies between components. We evaluate our methods in an environment similar to the the recently proposed Robo-Sumo task, where agents in a software physics simulator compete in tipping over their opponent or pushing them out of the arena. Our results show that the proposed methods are indeed capable of generating strong agents, significantly outperforming baselines that focus on optimizing the agent policy alone.
The Body is Not a Given: Joint Agent Policy Learning and Morphology Evolution
2019-05-17
In: Elkind, Edith and Veloso, Manuela, (eds.) Proceedings of the 18th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2019). (pp. pp. 1134-1142). IFAAMAS (International Foundation of Autonomous Agents and MultiAgent Systems): Montreal, Canada. (2019)
Paper
Electronic Resource
English
Transport policy in Australia—Evolution, learning and policy transfer
Elsevier | 2010
|Transport policy in Australia—Evolution, learning and policy transfer
Online Contents | 2011
|General guidelines given for tubular joint design (1)
Tema Archive | 1979
|Anti-drone Policy Learning Based on Self-attention Multi-agent Deterministic Policy Gradient
Springer Verlag | 2022
|