This paper introduces a new class of multi-agent discrete-time dynamic games, known in the literature as dynamic graphical games. For that reason a local performance index is defined for each agent that depends only on the local information available to each agent. Nash equilibrium policies and best-response policies are given in terms of the solutions to the discrete-time coupled Hamilton–Jacobi equations. Since in these games the interactions between the agents are prescribed by a communication graph structure we have to introduce a new notion of Nash equilibrium. It is proved that this notion holds if all agents are in Nash equilibrium and the graph is strongly connected. A novel reinforcement learning value iteration algorithm is given to solve the dynamic graphical games in an online manner along with its proof of convergence. The policies of the agents form a Nash equilibrium when all the agents in the neighborhood update their policies, and a best response outcome when the agents in the neighborhood are kept constant. The paper brings together discrete Hamiltonian mechanics, distributed multi-agent control, optimal control theory, and game theory to formulate and solve these multi-agent dynamic graphical games. A simulation example shows the effectiveness of the proposed approach in a leader-synchronization case along with optimality guarantees.


    Access

    Download


    Export, share and cite



    Title :

    Multi-agent discrete-time graphical games and reinforcement learning solutions


    Contributors:
    Abouheaf, M. (author) / Lewis, F.L. (author) / Vamvoudakis, K.G. (author) / Haesaert, S. (author) / Babuska, R. (author)

    Publication date :

    2014-01-01


    Remarks:

    Abouheaf , M , Lewis , F L , Vamvoudakis , K G , Haesaert , S & Babuska , R 2014 , ' Multi-agent discrete-time graphical games and reinforcement learning solutions ' , Automatica , vol. 50 , no. 12 , pp. 3038-3053 . https://doi.org/10.1016/j.automatica.2014.10.047



    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English


    Classification :

    DDC:    629



    Hierarchical reinforcement learning for real-time strategy games

    Niel, Remi / Krebbers, Jasper / Drugan, Madalina M. et al. | BASE | 2018

    Free access



    Experience generalization for multi-agent reinforcement learning

    Pegoraro, Renê / Costa, AHR / Ribeiro, CHC | BASE | 2001

    Free access