Parallel Transfer Learning in Multi-Agent Systems: What, when and how to transfer?
Adam Taylor, Ivana Dusparić, Maxime Guériau, Siobhán Clarke · 2019
Multi-agent Reinforcement Learning (RL) is frequently used in large-scale autonomous systems to learn the behaviours that best suit the system's operating environment. Learning can take a significant amount of time during which an RL system's performance is necessarily suboptimal. Transfer learning (TL), a method of reusing knowledge which has been gained in one task to improve the performance in another, has been used to speed up learning in single RL agent systems. TL requires learning on a source task to complete before transferring it to a target task, i.e., transfer is done offline. Parallel Transfer Learning (PTL), a technique which enables the source and target tasks to run concurrently, has been proposed to enable online transfers. However, the online selection of knowledge to be transferred, as well as online ways of integration of that knowledge on the receiving agents remain open issues. This paper proposes methods for selecting the knowledge to be transferred in PTL, frequency and size of transfers, and methods for knowledge integration into the target task. We evaluate the proposed approaches in two canonical RL examples: Mountain Car and Co-operative Predator Prey Pursuit. We show that PTL, similarly to RL, is highly sensitive to parameter selection and that suitable parameters differ per scenario.