Simulating Aggregation Algorithms for Empirical Verification of Resilient and Adaptive Federated Learning
Hongwei Jin, Ning Yan, Masood Mortazavi · 2020
Federated learning (FL) environments are dynamic: Users are recruited to participate, may drop out of participation, may vary in their availability and speed, and may suffer bandwidth variations. As such, production environments demand a resilient and adaptive design. Insisting on synchronous FL, with potentially millions or hundreds of thousands of clients, will lead to various issues-bursts in processing and communication loads, complicated procedures to handle laggards, and special protocols to manage client failures. In this paper, we focus on asynchronous, adaptive, and resilient operating environments for FL. We develop a simulation scheme and a set of associated aggregation algorithms as a method for investigating the soundness of asynchronous and adaptive system designs and operational principles. Our simulation model can capture the statistical impact of FL clients' intermittent operations. We also propose an aggregation algorithm usable when clients' participation and refresh rates vary. We suggest an approach for identifying valid and adaptive operating configurations including those that can be used to trade-off computational load and convergence speeds in FL programs.