This paper introduces a new class of multi-agent discrete-time dynamic games, known in the literature as dynamic graphical games. For that reason a local performance index is defined for each agent that depends only on the local information available to each agent. Nash equilibrium policies and best-response policies are given in terms of the solutions to the discrete-time coupled Hamilton–Jacobi equations. Since in these games the interactions between the agents are prescribed by a communication graph structure we have to introduce a new notion of Nash equilibrium. It is proved that this notion holds if all agents are in Nash equilibrium and the graph is strongly connected. A novel reinforcement learning value iteration algorithm is given to solve the dynamic graphical games in an online manner along with its proof of convergence. The policies of the agents form a Nash equilibrium when all the agents in the neighborhood update their policies, and a best response outcome when the agents in the neighborhood are kept constant. The paper brings together discrete Hamiltonian mechanics, distributed multi-agent control, optimal control theory, and game theory to formulate and solve these multi-agent dynamic graphical games. A simulation example shows the effectiveness of the proposed approach in a leader-synchronization case along with optimality guarantees.


    Zugriff

    Download


    Exportieren, teilen und zitieren



    Titel :

    Multi-agent discrete-time graphical games and reinforcement learning solutions


    Beteiligte:
    Abouheaf, M. (Autor:in) / Lewis, F.L. (Autor:in) / Vamvoudakis, K.G. (Autor:in) / Haesaert, S. (Autor:in) / Babuska, R. (Autor:in)

    Erscheinungsdatum :

    2014-01-01


    Anmerkungen:

    Abouheaf , M , Lewis , F L , Vamvoudakis , K G , Haesaert , S & Babuska , R 2014 , ' Multi-agent discrete-time graphical games and reinforcement learning solutions ' , Automatica , vol. 50 , no. 12 , pp. 3038-3053 . https://doi.org/10.1016/j.automatica.2014.10.047



    Medientyp :

    Aufsatz (Zeitschrift)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch


    Klassifikation :

    DDC:    629



    Hierarchical reinforcement learning for real-time strategy games

    Niel, Remi / Krebbers, Jasper / Drugan, Madalina M. et al. | BASE | 2018

    Freier Zugriff



    Experience generalization for multi-agent reinforcement learning

    Pegoraro, Renê / Costa, AHR / Ribeiro, CHC | BASE | 2001

    Freier Zugriff

    Multi Agent Reinforcement Learning for Gridworld Soccer Leadingpass

    Sari, Safreni Candra / Prihatmanto, Ary Setijadi / Kuspriyanto | Springer Verlag | 2013