This paper introduces a new class of multi-agent discrete-time dynamic games, known in the literature as dynamic graphical games. For that reason a local performance index is defined for each agent that depends only on the local information available to each agent. Nash equilibrium policies and best-response policies are given in terms of the solutions to the discrete-time coupled Hamilton–Jacobi equations. Since in these games the interactions between the agents are prescribed by a communication graph structure we have to introduce a new notion of Nash equilibrium. It is proved that this notion holds if all agents are in Nash equilibrium and the graph is strongly connected. A novel reinforcement learning value iteration algorithm is given to solve the dynamic graphical games in an online manner along with its proof of convergence. The policies of the agents form a Nash equilibrium when all the agents in the neighborhood update their policies, and a best response outcome when the agents in the neighborhood are kept constant. The paper brings together discrete Hamiltonian mechanics, distributed multi-agent control, optimal control theory, and game theory to formulate and solve these multi-agent dynamic graphical games. A simulation example shows the effectiveness of the proposed approach in a leader-synchronization case along with optimality guarantees.
Multi-agent discrete-time graphical games and reinforcement learning solutions
2014-01-01
Abouheaf , M , Lewis , F L , Vamvoudakis , K G , Haesaert , S & Babuska , R 2014 , ' Multi-agent discrete-time graphical games and reinforcement learning solutions ' , Automatica , vol. 50 , no. 12 , pp. 3038-3053 . https://doi.org/10.1016/j.automatica.2014.10.047
Article (Journal)
Electronic Resource
English
DDC: | 629 |