Multidimensional dynamic attention for multivariate time series forecasting
Sarah Almaghrabi, Mashud Rana, Margaret Hamilton, Mohammad Saiedur Rahaman · Applied Soft Computing · 2024
Attention-based models have been very effective in identifying important lagged variables for multivariate time series (MTS) forecasting applications. However, current attention-based models only provide static weights and do not consider the dynamic nature of predictions for multistep predictions of heterogeneous MTS. To address these limitations, this paper proposes a novel multidimensional dynamic attention (MDA) model for computing lagged variable importance. It incorporates a dynamic representation learner unit and considers multiple attention calculations to account for prediction dynamics, temporal information, and variable relations. Extensive experiments with both synthetic and real-world data demonstrate the effectiveness of the MDA model. It outperforms existing methods in the literature for sequence-to-sequence prediction of heterogeneous MTS in most cases and accurately identifies important features. MDA demonstrates enhancements up to 33% with real-world datasets. These findings demonstrate that the MDA model is a promising approach to MTS forecasting. The proposed attention mechanism can be utilised for other tasks related to MTS analysis beyond just forecasting, potentially enhancing the performance and interpretability of various MTS applications. The code is publicly available on GitHub, promoting widespread adoption and further research in this field https://github.com/sarah-almaghrabi/MDA-Multidimensional-Dynamic-Attention . • Proposes a novel Multidimensional Dynamic Attention (MDA) model for forecasting. • Utilises dynamic representation learning to handle heterogeneous data effectively. • Dynamic scoring considers prediction dynamics, temporal info, and variable relations. • Evaluates efficacy of sequential and parallel attention mechanisms in prediction. • Applies MDA to multiple datasets showcasing its relevance for practical application.