Transfer of Knowledge Across Tasks
Ricardo Vilalta, Mikhail M. Meskhi · Cognitive technologies · 2022
Abstract This area is often referred to as transfer of knowledge across tasks, or simply transfer learning; it aims at developing learning algorithms that leverage the results of previous learning tasks. This chapter discusses different approaches in transfer learning, such as representational transfer, where transfer takes place after one or more source models have been trained. There is an explicit form of knowledge transferred directly to the target model or to the meta-model. The chapter also discusses functional transfer, where two or more models are trained simultaneously. This situation is sometimes referred to as multi-task learning. In this approach, the models share their internal structure (or possibly some parts) during learning. Other topics include instance-, feature-, and parameter-based transfer learning, often used to initialize the search on the target domain. A distinct topic is transfer learning in neural networks, which includes, for instance, the transfer of a part of the network structure. The chapter also presents the double loop architecture, where the base-learner iterates over the training set in an inner loop, while the metalearner iterates over different tasks to learn metaparameters in an outer loop. Details are given on transfer learning within kernel methods and parametric Bayesian models.