Sparse Tabular Multiagent Q-learning ⁄
Jelle R. Kok, Nikos Vlassis · 2004
Multiagent learning problems can in principle be solved by treating the joint actions of the agents as single actions and applying singleagent Q-learning. However, the number of joint actions is exponential in the number of agents, rendering this approach infeasible for most problems. In this paper we investigate a sparse representation of the Q-function by only considering the joint actions in those states in which coordination is actually required. In all other states single-agent Q-learning is applied. This o#ers a compact state-action value representation, without compromising much in terms of solution quality. We have performed experiments in the predator-prey domain and compared our method to other multiagent reinforcement learning methods with promising results.