Dwtformer: Wavelet decomposition Transformer with 2D Variation for Long-Term Series Forecasting
Yujie Cao, Xi Juan Zhao · 2023
Benefiting from the boom in deep learning and natural language processing, RNNs, CNNs and Transformers have significantly improved the accuracy of multivariate long time series prediction, which focus on how to discover the long-term dependence of long time series and how to capture the overall trend of time series. But They ignore the complex intrinsic features of the series (the characteristics of intra-period and inter-period variations). Based on the observation of the multi-periodicity of time series, this study extends the analysis of time series to a higher space by decomposing a complex 1D time series into a set of 2D tensors based on multiple frequencies. Through this transformation, we connect the time series prediction to the computer vision so we can get more effective techniques which can be employed to extract complex temporal variations from the transformed 2D tensors. To address these issues, this paper proposes to combine the Transformer with a wavelet decomposition-based 2D feature learning module. The 2D feature learning module captures the complex period variations of the time series and the Transformer captures the long-term historical details. To more fully learn the periodic features, this paper proposes Dwtformer by referring to the auto-correlation mechanism in Autoformer. Experiments on four benchmark datasets show that compared to state-of-the-art methods, Dwtformer can reduce multivariate time series prediction errors by 14.9%.