Enhancing Global and Local Context Modeling in Time Series Through Multi-Step Transformer-Diffusion Interaction
Dagyeong Na, Jinho Kang, Byoungwoo Kang, Junseok Kwon · IEEE Access · 2025
Multivariate time-series forecasting (MTSF) has become crucial across various domains, with transformer-based architectures emerging as the primary choice because of their excellent ability to capture long-term dependencies. However, these models often fail to represent fine-grained local patterns, which are critical for accurate forecasting. To address this limitation, we propose a novel MTSF method based on the diffusion process, utilizing the transformer as prior knowledge. By conditioning the denoising process on global embeddings derived from the transformer, our approach effectively captures both global and local patterns. However, when transformers are used as conditional priors in the diffusion process, errors in their predictions can propagate, adversely affecting overall performance. Thus, we also introduce a mechanism that mitigates the impact of such errors by enabling effective interactions between transformers and diffusion models. Furthermore, the proposed multi-step denoising process progressively refines temporal patterns, preserving both global structures and local variations to enhance robustness and generalization. Experimental results demonstrate that the proposed approach overcomes the limitations of existing models, consistently outperforming previous state-of-the-art models (TMDM), achieving up to 97.9% lower mean squared error (MSE) on Weather and 96.3% on ETTm2.