Modeling and Optimizing Resource-Constrained Instance-Based Transfer Learning
Mohammad Askarizadeh, Mostafa Hussien, Alireza Morsali, Kim Khoa Nguyen · GLOBECOM 2022 - 2022 IEEE Global Communications Conference · 2022
Transfer learning (TL) reduces the training overheads by transferring knowledge across domains/tasks. However, the advantages of TL come with computation and communication costs. Therefore, the decision to transfer knowledge between learners should be optimized while at the same time avoiding negative transfer (NT), i.e. when the source information does not improve but rather degrades the learning performance in the target. In this paper, we propose a new notion namely, regret of learner (RoL) as a quantitative measure for the learner's performance, computation costs and communication resources of TL. Then, we use a convex combination of the empirical source and target errors with respect to the feasibility and resource constraints to design an optimization model called OPTL that deploys a TL model in a resource-constrained environment to avoid NT. This model can be employed as a general framework for different ML methods and various communication scenarios and use cases by changing the unification parameters. To validate our approach, we use OPTL for optimized TL in a deep learning (DL)-based classification problem. Extensive experiments confirm the efficiency of our proposed method.