SafeCool: Safe and Energy-Efficient Cooling Management in Data Centers With Model-Based Reinforcement Learning
Jianxiong Wan, Yanduo Duan, Xiang Gui, Chuyi Liu, Leixiao Li, Zhiqiang Ma · IEEE Transactions on Emerging Topics in Computational Intelligence · 2023
Optimizing the cooling system plays a central role for capping the data center power consumption. However, the performance of traditional cooling management strategies is not satisfactory due to the complexity of thermodynamic process. Recently, several works leveraged Reinforcement Learning (RL) to improve the energy efficiency of data center cooling system. While they demonstrated that it is possible to reduce the cooling power consumption via RL, there are still some key challenges that have to be addressed before field deployment, such as safe operation guarantee and sample complexity, etc. In this paper, we proposeSafeCool, an actor-critic Model-Based Reinforcement Learning (MBRL) algorithm for data center cooling management.SafeCoolincorporates two system models, i.e., a transition model to predict the future system state and a risk model to estimate the negative effect of executing an action. The safety of proposed algorithm is ensured by Model Predictive Control (MPC) and risk-guided exploration. In addition, by employing the MBRL framework,SafeCoolachieves higher sample efficiency and accelerated convergence. Simulations using real-world workload trace reveal thatSafeCoolsaves up to 13.18% cooling power compared with state-of-the-art MBRL data center cooling solutions.