Lightweight Spectrum Prediction Based on Knowledge Distillation
R. Cheng, J. Zhang, Jiaqi Deng, Yan Zhu · Radioengineering · 2023
To address the challenges of increasing complexity and larger number of training samples required for high-accuracy spectrum prediction, we propose a novel lightweight model, leveraging a temporal convolutional network (TCN) and knowledge distillation.First, the prediction accuracy of TCN is enhanced via a self-transfer method.Then, we design a two-branch network which can extract the spectrum features efficiently.By employing knowledge distillation, we transfer the knowledge from TCN to the two-branch network, resulting in improved accuracy for spectrum prediction of the lightweight network.Experimental results show that the proposed model can improve accuracy by 19.5% compared to the widelyused LSTM model with sufficient historical data and reduces 71.1% parameters to be trained.Furthermore, the prediction accuracy is improved by 17.9% compared to Gated Recurrent Units (GRU) in the scenarios with scarce historical data.