Model-Oriented Training With Two-Stage Hierarchical Knowledge Distillation Under Non-IID Conditions in Federated Edge–Cloud Collaboration
Yishan Chen, Wenshuo Dai, Junxiao Han, Zhen Qin, Shuiguang Deng · IEEE Transactions on Cloud Computing · 2025
With the continuous rolling-out of wireless edge cloud networks, Federated Learning (FL) has emerged as a promising solution for decentralized model training without exposing raw data. However, conventional centralized FL faces several limitations in resource-constrained mobile environments, including limited privacy-preserving capabilities and substantial communication overhead, which can lead to privacy leakage. Moreover, in non-independent and identically distributed (Non IID) data environments, FL faces the critical challenge of “client drift”, which leads to performance degradation. To address these challenges, this paper proposes TWHFL, a two-stage hierarchical knowledge distillation framework for Non-IID federated learning, designed to enhance terminal privacy protection and improve model personalization under heterogeneous data distributions. Specifically, in the cloud-edge collaboration stage, edge servers generate pseudo “hard samples” for all sub-MEC centers by optimizing noise inputs guided by feature distribution statistics (e.g., batch normalization running means and variances). To alleviate label distribution skew, both the label proportions and the volume of pseudo data are dynamically adapted based on the real-time operational state of each sub-MEC center. In the edge-terminal collaboration stage, each sub-MEC center conducts localized training using both real and synthetic data without external communication, thereby significantly reducing the risk of privacy leakage. Furthermore, a joint optimization problem is formulated to determine optimal configurations of pruning rates, CPU frequencies, up-link power, and bandwidth allocation, while jointly considering constraints on convergence rate, energy consumption, and latency. Experimental results show that the proposed TWHFL framework can effectively balance privacy protection and model performance in Non-IID settings.