Estimation of Distribution Algorithms for Solving Reinforcement Learning Problems
Hisashi Handa · IEEJ Transactions on Electronics Information and Systems · 2010
Estimation of Distribution Algorithms (EDAs) are a promising evolutionary computation method. Due to the use of probabilistic models, EDAs can outperform conventional evolutionary computation. In this paper, EDAs are extended to solve reinforcement learning problems which are a framework for autonomous agents. In the reinforcement learning problems, we have to find out better policy of agents such that it yields a large amount of reward for the agents in the future. In general, such policy can be represented by conditional probabilities of agents' actions, given the perceptual inputs. In order to estimate such a conditional probability distribution, Conditional Random Fields (CRFs) by Lafferty (2001) are introduced into EDAs. The reason why CRFs are adopted is that CRFs are able to learn conditional probabilistic distributions from a large amount of input-output data, i.e., episodes in the case of reinforcement learning problems. Computer simulations on Probabilistic Transition Problems and Perceptual Aliasing Maze Problems show the effectiveness of EDA-RL.