Function Approximation Using Tile and Kanerva Coding For Multi-Agent Systems
Cheng Feng Wu, Waleed M. Meleis · 2009
Function approximation can improve the ability of a reinforcement learner. Tile coding and Kanerva coding are two classical methods for implementing function approximation, but these methods may give poor performance when applied to large-scale, high-dimensional instances. In the paper, we evaluate a collection of hard instances of the predator-prey pursuit problem, a classic multi-agent reinforcement learning problem, to compare these two methods and their optimization techniques. We first show that Kanerva coding gives better results than Tile coding when the dimension of the instances increases. We then describe a feature optimization mechanism and show that it can increase the fraction of solved instances by both Tile coding and Kanerva coding. Finally, we demonstrate that a fuzzy approach to function approximation can further increase the fraction of instances. We show that our fuzzy approach to Kanerva coding outperforms fuzzy Tile coding when feature optimization is applied. on large-scale, high-dimensional multi-agent problems.