Hierarchical and Heterogeneous Federated Learning via a Learning-on-Model Paradigm
Leming Shen, Qiang Yang, Kaiyan Cui, Yuanqing Zheng, Xiao-Yong Wei, Jianwei Liu, Jinsong Han · IEEE Transactions on Mobile Computing · 2025
Federated Learning (FL) collaboratively trains a shared global model without exposing clients' private data. In practical FL systems, clients (e.g., smartphones and wearables) typically have disparate system resources. Traditional FL, however, adopts a one-size-fits-all solution, where a homogeneous large model is sent to and trained on each client. This method results in an overwhelming workload for less capable clients and starvation for others. To tackle this, we proposeFedConv, a client-friendly FL framework, minimizing the system overhead on resource-constrained clients by providing heterogeneous customized sub-models.FedConvfeatures a novellearning-on-modelparadigm that learns the parameters of heterogeneous sub-models viaconvolutional compression. To aggregate heterogeneous sub-models, we proposetransposed convolutional dilationto convert them back to large models with a unified size while retaining personalized information. The compression and dilation processes, transparent to clients, are tuned on the server using a small public dataset. We further propose ahierarchical and clustering-based local trainingstrategy for enhanced performance. Extensive experiments on six datasets show thatFedConvoutperforms state-of-the-art FL systems in terms of model accuracy (by more than 35% on average), computation and communication overhead (with 33% and 25% reduction, respectively).