Best response learning based on Gaussian regression for multi-agent systems in continuous spaces
Haijun Wei, Xin Chen, Min Wu, Weihua Cao · Chinese Control Conference · 2012
In the implementations of multi-agent systems, generalization is always viewed as one of the key issues before multi-agent reinforcement learning algorithms are applicable to continuous environments. The paper proposes a best response learning based on Gaussian regression for multi-agent systems in continuous spaces. With a new Q value with reduced dimension defined, the algorithm entitles agent to learning strategy adapting to others' behaviors. To realize generalization, probabilistic model of state transition in the algorithm is constructed by using Gaussian regression, so that dynamic programming can be applied directly to generate the best response strategy. And both Q-function model and V-function model are built real time in order to generalize state and action spaces. Thus the learning agent is able to tracking partners' strategies. In the simulation of Double-cart-pole, which is a typical coordinated control problem, even if dynamics is unknown in priori, the algorithm enables agent to learn coordinated strategy, and realize generalization of state space as well.