DiPLe: Learning Directed Collaboration Graphs for Peer-to-Peer Personalized Learning
Xue Zheng, Parinaz Naghizadeh, Aylin Yener · 2022 IEEE Information Theory Workshop (ITW) · 2022
We study fully decentralized learning in which agents learn collaborative, yet personalized prediction models. Specifically, when learners’ local datasets are non-IID, a collaboratively trained global model (such as those learned through most federated learning algorithms to minimize the sum of losses across all agents) may sacrifice the local performance on agents’ private datasets. To address this issue and enable personalized learning, we propose DiPLe : an algorithm for Directed Personalized Learning. Through our algorithm, each agent identifies "relevant" agents with whom to exchange model information. This leads to a weighted and directed collaboration graph. Agents repeatedly update this graph, and then exchange information with neighboring agents on this learned graph, to collaboratively train their personalized models. We provide analytical results on the generalization error bounds and convergence of our proposed learning method. We verify the performance of DiPLe through numerical experiments, and show its advantages in terms of personalization compared to a number of existing federated learning and personalized learning algorithms.