CIPformer: Channel-Independence Integrated with Patching Transformer for Multivariate Long-Term Time Series Forecasting
Peng Qiu, Jinhong Li · 2024
Although the Transformer-based models have been the dominant architecture for long-term time series forecasting tasks in recent years, there is an essential problem. Most of these methods adopt the standard self-attention mechanism which not only has a high computational complexity, but also fails to capture the local and global dependencies from historical time series data simultaneously. In this paper, we propose CIPformer, a novel Transformer-based model for long-term time series forecasting. It utilizes channel independence, where each channel comprises a solitary univariate time series, sharing identical embedding and Transformer weights across all series. Additionally, patch embedding is employed to extract the local information and reduce time and memory complexity of the attention mechanism quadratically. With a deftly designed patch-wise attention which encompasses both intra-patch and inter-patch attention, CIPformer empowers the Transformer encoder with the ability to capture both local features and global interactions. Furthermore, in order to learn linear information, we employ a linear model to better capture linear trends and details. Extensive experiments conducted on six real-world datasets have demonstrated that our proposed model CIPformer can enhance the multivariate long-term time series forecasting performance, when compared with some of the state-of-the-art Transformer-based methods.