Efficient Offline Communication Policies for Factored Multiagent POMDPs
João V. Messias, Matthijs T. J. Spaan, Pedro U. Lima · 2011
Factored Decentralized Partially Observable Markov Decision Processes (Dec-POMDPs) form a powerful framework for multiagent planning under uncertainty, but optimal solutions require a rigid history-based policy representation. In this paper we allow inter-agent communication which turns the problem in a central-ized Multiagent POMDP (MPOMDP). We map belief distributions over state fac-tors to an agent’s local actions by exploiting structure in the joint MPOMDP pol-icy. The key point is that when sparse dependencies between the agents ’ decisions exist, often the belief over its local state factors is sufficient for an agent to un-equivocally identify the optimal action, and communication can be avoided. We formalize these notions by casting the problem into convex optimization form, and present experimental results illustrating the savings in communication that we can obtain. 1