TSFormer: Temporal-Aware Transformer for Multi-Horizon Forecasting with Learnable Positional Encodings and Attention Mechanisms
Franck Junior Aboya Messou, Jinhua Chen, Tong Liu, Shilong Zhang, Keping Yu · 2025
Transformer models have significantly advanced time-series forecasting by effectively capturing long-range dependencies in IoT-enabled environments. However, challenges persist in handling temporal irregularities, integrating explicit temporal features, and refining attention mechanisms to enhance forecasting accuracy and efficiency. This paper introduces TSFormer, a novel Transformer-based framework incorporating learnable positional encodings and explicit temporal covariates to improve temporal representations and adaptability. Notably, learnable encodings achieve global reductions of$\mathbf{1 7. 8 \%}$in MSE and 10.9% in MAE on the IHPC dataset compared to fixed sinusoidal approaches. Additionally, we incorporate an attention mechanism that leverages masked and cross-attention to enhance multihorizon forecasting. Experiments on two benchmark datasets demonstrate that TSFormer achieves improved forecasting performance while maintaining robustness and interpretability, making it well-suited for real-world IoT-based applications.