A Transformer Model Incorporating Dynamic Chunking Strategy for Multivariate Time Series Classification
Yiyang Wang, Shijia Li, Ziyuan Cheng, Zhongheng Sun, Cuiling Jiang, Yongjing Wan · 2024
Multivariate time series classification tasks play a crucial role in the field of data mining and find wide applications in areas such as audio, healthcare, and transportation. The core challenge in multivariate time series classification tasks lies in capturing the relationships between variables and effectively learning the intricate details within the sequences. In recent years, multivariate time series classification methods based on deep learning have continued to emerge and have achieved certain results. However, existing methods commonly suffer from high computational complexity and an inability to capture the hidden relationships between variables fully. To address these issues, this paper proposes a method named DCSformer for multivariate time series classification, based on the Transformer framework. To reduce computational complexity, DCSformer utilizes a dynamic chunking strategy, processing sequences in patch form. It dynamically adjusts the chunk sizes and positions based on the internal feature values within the sequence. This strategy aims to not only decrease computational complexity but also better ensure the integrity of crucial sequence information during chunking. To comprehensively capture potential associations among variables, DCSformer utilizes a multivariate information fusion method, processing multivariate sequences dimension by dimension. It learns the positional relationships among various variables through the network and embeds these relationships into self-attention mechanisms to adjust weights, thereby fully considering and integrating interactions among variables. A series of experiments on electrocardiogram signals, electroencephalogram signals, human activity signals, and traffic signals in the UEA dataset were conducted in this study, comparing against baseline models. Experimental results demonstrate that DCSformer achieves superior classification performance across seven datasets in the evaluation.