Q-Learning with Side Information in Multi-Agent Finite Games
Mathieu Sylvestre, Lacra Pavel · 2019
In this paper, we propose a discrete-time Nash equilibrium-seeking reinforcement learning algorithm for n-player finite games which can be adapted to leverage side information when it is available. Under standard assumptions used in stochastic approximation theory, the induced strategies of the discrete-time process converge to a perturbed Nash equilibrium for a class of games characterized by negative monotonicity properties of its utility, which encompasses 2-player zero-sum games and concave potential games. Our convergence results are still valid when no side information is exploited. Through numerical simulations of certain representative games, we show that a faster rate of convergence to a Nash equilibrium can be achieved when more side information is available and exploited.