Learning automata based multi‐agent system algorithms for finding optimal policies in Markov games
Behrooz Masoumi, Mohammad Reza Meybodi · Asian Journal of Control · 2010
Abstract Markov games, as the generalization of Markov decision processes to the multi‐agent case, have long been used for modeling multi‐agent systems (MAS). The Markov game view of MAS is considered as a sequence of games having to be played by multiple players while each game belongs to a different state of the environment. In this paper, several learning automata based multi‐agent system algorithms for finding optimal policies in Markov games are proposed. In all of the proposed algorithms, each agent residing in every state of the environment is equipped with a learning automaton. Every joint‐action of the set of learning automata in each state corresponds to moving to one of the adjacent states. Each agent moves from one state to another and tries to reach the goal state. The actions taken by learning automata along the path traversed by the agent are then rewarded or penalized based on the comparison of the average reward received by agent per move along the path with a dynamic threshold. In the second group of the proposed algorithms, the concept of entropy has been imported into learning automata based multi‐agent systems to improve the performance of the algorithms. To evaluate the performance of the proposed algorithms, computer experiments have been conducted. The results of experiments have shown that the proposed algorithms perform better than the existing algorithms in terms of speed and accuracy of reaching the optimal policy. Copyright © 2010 John Wiley and Sons Asia Pte Ltd and Chinese Automatic Control Society