Hierarchical Multi-Task Learning: a Cascade Approach Based on the Notion of Task Relatedness
Alon Zweig, Daphna Weinshall · 2013
Multi-task learning can be shown to improve the generalization performance of single tasks under certain conditions. Typically, the algo-rithmic and theoretical analysis of multi-task learning deals with a two-level structure, in-cluding a group of tasks and a single task. In many situations, however, it is benefi-cial to consider varying degrees of relatedness among tasks, assuming that some tasks are closely related and should contribute more to the learning process, while other tasks are less related but can still contribute some in-formation to the learning process. The exten-sion of current approaches to the multi-level setting may not be trivial. We propose a general framework for a full hi-erarchical multi-task setting. We define an explicit notion of hierarchical tasks related-ness, where at each level we assume that some aspects of the learning problem are shared. We suggest a cascade approach, where at each level of the hierarchy a learner learns jointly the uniquely shared aspects of the tasks by finding a single shared hypothesis. This shared hypothesis is used to bootstrap the preceding level in the hierarchy, forming a hypothesis search space. We analyze suf-ficient conditions for our approach to reach optimality, and provide generalization guar-antees in an empirical risk minimization set-ting. 1.