Multivariate Time Series Prediction based on Improved Transformer Model in Computing System
Menghao Gong, Jianpeng Sun, Xiguo Xie, Yu Zheng · 2023
In IT operations and maintenance applications within computing systems, monitored data is often modeled as Multivariate Time Series (MTS). Predicting MTS has been extensively studied, and various models, including statistical algorithms and deep learning networks, have been proposed to capture multidimensional and nonlinear features. However, current models suffer from several issues. Firstly, for data with multiple dimensions in computer systems, existing models do not sufficiently and explicitly explore and exploit cross-dimensional dependencies. Secondly, there is a demand for real-time processing of large volumes of data in IT operations and maintenance applications within computing systems, and current deep learning network models incur high computational costs. Finally, the temporal characteristics of time series data are not effectively utilized. To address these challenges, we propose a Transformer-based time series prediction model. Building upon the current model, we introduce a 2D Transformer model to extract cross-dimensional features, utilize a sparse attention mechanism to reduce computational costs, and introduce a novel positional encoding to replace traditional positional encoding. We evaluate the performance on real-world datasets from actual applications within computing systems. Significant correlations between prediction results and actual outcomes are observed. The current model is implemented and applied within the computing system at Zhengzhou University. We compare our model with several state-of-the-art baseline methods, and empirical results demonstrate that our model achieves superior performance.