Fixed vs. Dynamic Sub-Transfer in Reinforcement Learning.
James L. Carroll, Todd Peterson · 2002
We survey various task transfer methods in Qlearning and present a variation on fixed sub-transfer which we call dynamic sub-transfer. We discuss the benefits and drawbacks of dynamic sub-transfer as compared with the other transfer methods, and we describe qualitatively the situations where this method would be preferred over the fixed version of sub-transfer. We test this method against several other transfer methods in a simple three room grid world where portions of the source's policy are relevant to the target task and other portions are not. In this situation we found that dynamic sub-transfer converged to the optimal solution, avoiding the suboptimality inherent in fixed sub-transfer, while also avoiding some of the convergence problems often experienced by fixed sub-transfer.