TS2V: A Transformer-Based Siamese Network for Representation Learning of Univariate Time-Series Data
Chengyang Ye, Qiang Ma · 2022 IEEE 25th International Conference on Computer Supported Cooperative Work in Design (CSCWD) · 2022
In this study, we propose a Transformer-based Siamese network for learning the representation of univariate time-series data. Based on the characteristics of the time-series downstream task, the similarity measurement was introduced as the task for the pre-training model. Meanwhile, as the feature of univariate time-series is not easily extracted by the attention mechanism, a novel method to initialize the embedding vectors was proposed to emphasize the trend information of the input vector. The experimental results demonstrate that our model works well on classification tasks of univariate time-series with strong trend characteristics.