ACyLeR: An Enhanced iTransformer for Long-Term Time-Series Forecasting Using Adaptive Cycling Learning Rate
Mustafa Kamal, Ary Mazharuddin Shiddiqi, Ervin Nurhayati, Andika Laksana Putra, Farrela Ranku Mahhisa · 2024
Long-term time-series forecasting is critical in nu-merous domains, including economics, climate modeling, and energy management. Traditional deep learning models often struggle with optimizing hyperparameters, which can lead to suboptimal performance and increased sensitivity to initial conditions. This research addresses the problem by proposing an enhanced iTransformer model that integrates an Adaptive Cycling Learning Rate (ACLR) mechanism, named ACyLeR. The ACLR algorithm dynamically adjusts the learning rate during the training phase for better convergence and generalization while minimizing the risk of overfitting. The experiments were written in Python and tested using univariate Water Supply in Melbourne (WSM) and multivariate exchange rate (ER) datasets with 70% training, 10% validation, and 20% testing data grouping. Experimental results demonstrate that the ACyLeR with ACLR outperforms existing baseline models by achieving lower loss values and higher accuracy. The results significantly advance time-series forecasting using iTransformer.