Computing power network dynamic resource scheduling integrating time series mixing dynamic state estimation and hierarchical reinforcement learning
Hong Liu, Shulei Zhang, Li Li, Tingting Sun, Wei Xue, Xiaoxia Yao, Yong Kang Xu · Scientific Reports · 2026
With the evolution of cloud computing towards a multi-cloud architecture, cross-cloud resource scheduling faces challenges such as heterogeneous environment adaptation and slow dynamic load response. How to improve resource utilization while ensuring service quality has become a core challenge in the field of cloud management. To address this need, we propose the TSL-HRL intelligent scheduling framework, which integrates time-series feature modeling and hierarchical reinforcement learning. The framework utilizes a time-series mixing module to deeply mine the periodic fluctuations and burst demand features of computing, storage, and network resources. It integrates a dynamic state estimation module with Kalman filtering to capture real-time changes in resource supply and demand. Additionally, it constructs a high-level planning - low-level response hierarchical reinforcement learning architecture: the high-level Q-learning algorithm formulates a global long-term resource allocation strategy to ensure optimal overall scheduling, while the low-level A2C algorithm adjusts the execution plan based on real-time network fluctuations and node load, enabling fast adaptation to dynamic changes, forming a macro-micro collaborative decision mechanism. In experiments on the Multi-Cloud Service Composition Dataset and Google 2019 Cluster dynamic node scenarios, TSL-HRL effectively balanced resource utilization efficiency and scheduling real-time performance with its three-level architecture design of time-series feature extraction - dynamic state perception - hierarchical strategy optimization. The study shows that TSL-HRL provides a systematic solution for resource management in multi-cloud environments. Future research will focus on lightweight extensions for edge-cloud collaborative scenarios, multi-objective energy consumption optimization frameworks, and meta-learning-driven rapid adaptation technologies, promoting the application and generalization of intelligent resource scheduling technologies in real-world complex scenarios.