Transferring previously learned back-propagation neural networks to new learning tasks
Lorien Pratt · 1993
When people learn a new task, they often build on their ability to solve related problems. For example, a doctor moving to a new country can use prior experience to aid in diagnosing patients. A chess player can use experience with one set of end-games to aid in solving a different, but related, set. However, although people are able to perform this sort of skill transfer between tasks, most neural network training methods in use today are unable to build on their prior experience. Instead, every new task must be learned from scratch. This dissertation explores how a back-propagation neural network learner can build on its previous experience. We present an algorithm, called Discriminability-Based Transfer (DBT), that facilitates the transfer of information from the learned weights of one network to the initial weights of another. Through evaluation of DBT on several benchmark tasks we demonstrate that it can speed up learning on a new task. We also show that DBT is more reliable than simpler methods for transfer.