Federated Learning Communications Optimization Using Sparse Single-Layer Updates
Rachid El Mokadem, Yann Ben Maissa, Zineb El Akkaoui · Procedia Computer Science · 2024
Federated learning has emerged as a robust framework for distributed machine learning, enabling model training across decentralized data sources while preserving data privacy. Despite its advantages, a persistent challenge remains: the high communication overhead during the model update process results in a high energy consumption on the client devices. This paper introduces a new approach that combines (1) weights sparsification technique with (2) single layer update of the shared neural network model. Our proposal serves dual purposes: it significantly reduces the volume of data transmitted during each training round while lessening the computational burden on resource-limited devices. Through empirical evaluations, we witness an impressive reduction—up to 98.3%—in data exchange during the aggregation phase without much compromising the model's performance. Moreover, we find that the communication cost savings scale with the size of the model, making our approach particularly advantageous for large, complex models. This work opens the door to for more energy-efficient and scalable federated learning implementations, especially in resource-constrained environments like IoT and mobile devices.