Federated Learning Meets Network Coding: Efficient Coded Hierarchical Federated Learning
Tianli Gao, Jiahong Lin, Congduan Li, Chee Wei Tan, Jun Gao · 2024
Federated learning is a machine learning framework that facilitates training a shared model from distributed clients. However, challenges persist in optimizing communication efficiency. In this paper, we focus on hierarchical federated learning and model its global aggregation as a network function computation problem, where the central server desires to compute the arithmetic sum of the clients' gradients. Inspired by network coding, we propose two Coded Hierarchical Federated Learning (CHFL) approaches to enhance communication efficiency. The first approach, Separated CHFL (S-CHFL), involves transmitting divided segments separately to relays using a greedy algorithm. We establish the upper and lower bounds of the computing rate, showing that S-CHFL can achieve perfect balance in reverse combination network and reach the upper bound in certain networks. The second approach is Mixed CHFL (M-CHFL) where divided segments are mixed into linear combinations for transmission. We show that M-CHFL may be more efficient when data comes from a sufficiently large alphabet and analyze its upper bound for the computing rate.