Why co-evolution beats temporal difference learning at Backgammon for a linear architecture, but not a non-linear architecture
Paul J. Darwen · 2002
No Free Lunch theorems show that the algorithm must suit the problem. This does not answer the novice's question: for a given problem, which algorithm to use? This paper compares co-evolutionary learning and temporal difference learning on the game of Backgammon, which (like many real-world tasks) has an element of random uncertainty. Unfortunately, to fully evaluate a single strategy using undirected sampling of board positions, using only random dice rolls, requires a great deal of computation. Evolution's all-or-nothing replacement of entire solutions needs accurate evaluation, but relatively rare board positions are needed to train above a certain level. Temporal difference learning, with its incremental changes, does not use such an all-or-nothing approach. These results have relevance to a variety of real-world tasks with uncertainty, such as schedule optimization.