Enhancing Collaborative Machine Learning in Resource-Limited Networks Through Knowledge Distillation and Over-the-Air Computation
Yue Zhang, Guopeng Zhang, Kun Yang, Yao Wen, Kezhi Wang · IEEE Transactions on Mobile Computing · 2025
Conventional collaborative machine learning (CML) faces significant challenges in resource-constrained environments, such as emergency scenarios with limited power, bandwidth, and computing resources, leading to increased communication delays and energy consumption. To address these issues, this paper introducesAir-CoKD, a novel CML framework designed to reduce resource consumption and training latency while preserving model performance.Air-CoKDleverages knowledge distillation (KD) to minimize data transmission by avoiding the direct sharing of model parameters. It also integrates over-the-air computation (AirComp) to aggregate local logits, optimizing bandwidth utilization. To address the dimensional differences in local logits caused by the unbalanced device data class,Air-CoKDemploys orthogonal frequency division multiplexing (OFDM) to transmitting local logits for different target classes. To handle aggregation errors introduced by AirComp, we conduct a detailed analysis of error bounds. Specifically, we convert the Kullback-Leibler (KL) divergence, used in KD loss function, into a quadratic upper bound for precise error quantification and effective optimization. Based on these insights, we propose a strategy to manage bandwidth constraints, transmission power limits, and device energy budgets withinAir-CoKD. Extensive simulations demonstrate thatAir-CoKDsurpasses state-of-the-art methods, effectively balancing training efficiency and model performance. The framework proves to be a robust solution for CML in resource-constrained networks.