Seq2SeqT3DT: a novel tolerance aware sequence to sequence tree model with three-dimensional input for time series regression
Aristeidis Mystakidis, Christos Tjortjis · International Journal of Machine Learning and Cybernetics · 2026
Multi-step ahead Time Series Forecasting (TSF) is crucial in domains such as energy management, transportation planning, and weather prediction. Traditional methods often rely on a direct strategy with multiple models or rolling/recursive strategies to predict future values. This can cause development difficulties, increase computational complexity and accumulate errors in specific time steps. In addition, existing sequence-to-sequence (Seq2Seq) models, although very powerful in several domains, can have complex structure with limited interpretability and difficulty to fit tabular TSF data. This paper introduces Seq2SeqT3DT, a novel tolerance-aware Seq2Seq tree model with three-dimensional input, a Decision Tree (DT) designed for multi-step forecasting regression tasks. Unlike conventional DTs, it is capable of directly mapping input sequences of historical data to output sequences of future predictions, within a single model framework. It extends the well-known DT algorithm and builds upon the baseline Seq2SeqDT (also known as multi-output or multi-target DT) by handling multi-dimensional outputs, allowing each leaf node to store a sequence of values, rather than a single prediction. It also features a more robust mechanism for tolerating prediction error through per-step or weighted tolerances, incorporating the option to use temporal derivatives, and applies temporal decay to favor recent features during splitting. We evaluate its performance by comparing it with the baseline Seq2SeqDT and the two established Seq2Seq Deep Learning (DL) models: a Long Short-Term Memory neural network (Seq2SeqLSTM) and a Seq2Seq transformer (Seq2SeqTF), on three forecasting datasets, for energy generation (EGF), energy load (ELF) and traffic congestion prediction (TCP), respectively, on 6-step and 12-steps ahead tasks. Experimental results demonstrate that, in the EGF 12-step task, Seq2SeqT3DT achieved average MAE = 0.1440, RMSE = 0.2800 and $$R^{2}$$ = 0.8619, outperforming all baselines, while in the EGF 6-step task showcased the lowest average RMSE (0.25) and highest $$R^{2}$$ (0.8898). On ELF, all models showed similar performance, with our model illustrating the lowest RMSE (0.2082) for the 12-step task. For the TCP task, our Seq2SeqT3DT was the second-best performer after Seq2SeqTF. These results show that Seq2SeqT3DT achieves better or competitive performance than Seq2SeqDT and Seq2Seq DL models in most cases, offering a promising alternative for multi-step TSF. Seq2SeqT3DT combines the simplicity and interpretability of DTs with the ability to handle complex Seq2Seq relationships, effectively bridging the gap between traditional DTs and modern Seq2Seq DL approaches.