Adaptive federated learning algorithm based on evolution strategies
茂果 公, 原 高, 炯乾 王, 元侨 张, 善峰 王, 飞 谢 · Scientia Sinica Informationis · 2022
Federated learning is a deep learning technique that ensures data privacy with multiple device participation by training a globally shared model while storing private data locally. However, in a complex Internet-of-Things (IoT) environment, federated learning faces challenges of statistical heterogeneity and systematic heterogeneity. Because of different local data distributions and high communication costs, over-parameterized models are unsuited for direct deployment in IoT applications. Moreover, nonindependent, identically distributed data make federated learning with parameter-averaging aggregation more difficult to converge. Determining how to build personalized lightweight models for each client based on individual data and then aggregate these models has become a research problem with regard to federated learning. To solve this problem, we propose an adaptive federated learning algorithm based on an evolution strategy. The method regards each participant as an individual by encoding the model architecture through an evolution strategy, and it can adaptively generate a different customized subnet for each client through global optimization. According to the importance of the network unit and genotype, clients extract the corresponding subnets from the server-side supernets and perform local updates, which naturally fits the idea of the dropout. Extensive experiments on real-world datasets demonstrate that the proposed framework considerably improves the model performance compared with conventional federated learning. In particular, when the local data is not independent and uniformly distributed, the framework facilitates clients with limited communication bandwidths and computing power to participate in federated learning; the generalization ability of the global model is improved.