Towards Hierarchical Clustered Federated Learning With Model Stability on Mobile Devices
Biyao Gong, Tianzhang Xing, Zhidan Liu, Wei Xi, Xiaojiang Chen · IEEE Transactions on Mobile Computing · 2023
Clustered federated learning (CFL) has proved to be an effective way to alleviate the non-IID (not independently and identically distributed) data challenge, which severely restricts the wider application of federated learning. However, existing approaches either lack adaptability,i.e., they require an additional number of clusters as a guide when clustering, or lack effectiveness in terms of communication. In this paper, we explore the differences in the ability of different layers in a model to represent non-IID data, and propose a hierarchical CFL approach, namedHiCFL, which considers both adaptivity and communication efficiency. The improvement of communication efficiency is due to our proposed novel concept of model stability, which characterizes the variation of model weights during training. Based on model stability,HiCFLcan find the proper time to bi-partition the clusters of mobile devices in a hierarchical manner more quickly. We conduct extensive experiments based on popular datasets with various non-IID data settings. The results show thatHiCFLachieves excellent performance effectiveness and efficiency. Compared to state-of-the-art approaches,HiCFLcan improve the model accuracy by$2.0\% \sim 9.0\%$, while reducing the communication overheads by$27.3\% \sim 80.6\%$.