Spatiotemporal Sequence Prediction Based on Spatiotemporal Self-Attention Mechanism
Yuan Zhao, Junlin Lu · International Journal of Computers Communications & Control · 2024
This paper introduces the GCN-Transformer model, an innovative approach that combines Graph Convolutional Networks (GCNs) and Transformer architectures to enhance spatiotemporal sequence prediction. Targeted at applications requiring precise analysis of complex spatial and temporal data, the model was tested on two distinct datasets: PeMSD8 for traffic flow and KnowAir for air quality monitoring. The GCN-Transformer demonstrated superior performance over traditional models such as LSTMs, standalone GCNs, and other GCN-hybrid models, evidenced by its lower Root Mean Squared Error (RMSE) and Mean Absolute Error (MAE). An ablation study confirmed the importance of each component within the model, showing that removing elements like GCN layers, Transformer layers, attention mechanisms, or positional encoding detrimentally impacts performance. Overall, the GCN-Transformer model offers significant theoretical and practical contributions to the field of spatiotemporal data analysis, with potential applications across traffic management, environmental monitoring, and beyond.