BTFormer: A BNN-Based Trend-Aware Time-Series Prediction Model for Industrial Intelligence
Lei Ren, Shixiang Li, Xiaokang Wang, Haiteng Wang, Yuanjun Laili · IEEE Transactions on Industrial Informatics · 2024
Prediction of industrial time-series is crucial for various Industrial Internet of Things applications. Despite the high accuracy of deep learning methods for time-series prediction, the significant memory requirements of deep learning models pose a challenge for the limited computational resources of industrial edge devices. To address this issue, this work proposes BTFormer, which achieves a high compression rate while maintaining competitive performance. First, a binary adaptive attention module is proposed to mitigate the loss of attention information caused by binarization. Second, a trend information soft-link is proposed to propagate trend information between layers and improve the representation ability of the model. Finally, a distribution-guided distillation strategy is proposed to optimize the training process. The experiments demonstrate that BTFormer effectively reduces model memory usage by 31.0 times and improves computational efficiency by 32.8 times while maintaining competitive performance.