LightCTS*: Lightweight Correlated Time Series Forecasting Enhanced With Model Distillation
Zhichen Lai, Dalin Zhang, Huan Li, Christian S. Jensen, Hua Cai Lu, Yan Li Zhao · IEEE Transactions on Knowledge and Data Engineering · 2024
Correlated time series (CTS) forecasting is essential in many practical applications, such as traffic management and server load control. Various deep learning based solutions have been proposed to improve forecasting accuracy. However, while models have become increasingly computationally intensive, they struggle to improve accuracy. This study aims instead to enable more lightweight, accurate models suitable for resource-constrained devices. To achieve this goal, we characterize popular CTS forecasting models, yielding two observations for developing lightweight CTS forecasting. On this basis, we propose theLightCTSframework that adopts plain stacking of temporal and spatial operators instead of alternate stacking which is much more computationally expensive. Moreover,LightCTSfeatures light temporal and spatial operators, L-TCN and GL-Former, offering improved computational efficiency without compromising their feature extraction capabilities.LightCTSalso encompasses a last-shot compression scheme to reduce redundant temporal features and speed up subsequent computations. Next, we equipLightCTSwith two knowledge distillation modules,TafdandCaad, that result inLightCTS$^\star$retaining the original benefits ofLightCTS, while also being able to adapt to varying levels of ultra-constrained resources. Experimental studies offer detailed insight into these proposals and provide evidence that bothLightCTSandLightCTS$^\star$are capable of nearly state-of-the-art accuracy at substantially reduced computational costs.