Lifelong Learning of Discriminative Representations.
Ouais Alsharif, Philip Bachman, Joëlle Pineau · arXiv (Cornell University) · 2014
We envision a machine learning service provider facing a continuous stream of problems with the same input domain, but with output domains that may differ. Clients present the provider with problems implicitly, by labeling a few example inputs, and then ask the provider to train models which rea-sonably extend their labelings to novel inputs. The provider wants to avoid constraining its users to a set of common la-bels, so it does not assume any particular correspondence be-tween labels for a new task and labels for previously encoun-tered tasks. To perform well in this setting, the provider needs a representation of the input domain which, in expectation, permits effective models for new problems to be learned effi-ciently from a small number of examples. While this bears a resemblance to settings considered in previous work on mul-titask and lifelong learning, our non-assumption of inter-task label correspondence leads to a novel algorithm: Lifelong Learner of Discriminative Representations (LLDR), which explicitly minimizes a proxy for the intra-task small-sample generalization error. We examine the relative benefits of our approach on a diverse set of real-world datasets in three sig-nificant scenarios: representation learning, multitask learning and lifelong learning. 1