A Data-driven Analysis of a Cloud Data Center: Statistical Characterization of Workload, Energy and Temperature
Shashikant Ilager, Adel N. Toosi, Mayank Raj Jha, Ivona Brandić, Rajkumar Buyya · 2023
To efficiently manage large-scale cloud data centers, it is critical to understand data centers' workload, energy, and thermal characteristics and their impact on the system through data-driven analysis. However, most publicly available traces solely focuses on application workloads ignoring energy and thermal aspects, forcing existing studies to rely on inaccurate and unrealistic analytical or simulation models. In this paper, we present a comprehensive data-driven analysis of a production cloud data center. We monitor and collect the physical machine-level metrics such as resource utilization, energy, and temperature for up to nine months, with a system size of over 26000 CPU cores hosting, on average 1300 virtual machines. We perform a systematic statistical analysis to characterize the monitored data, and study their distributions, variations, trends, and inter-dependencies. We also develop data-driven models to predict resource usage and energy consumption of a physical machine, and demonstrate the usefulness of our dataset through this use case. Our study reveals interesting insights into the energy and thermal phenomena of a data center. The outcome of this analysis helps to increase infrastructure efficiency and long-term strategic planning and improve key business KPIs. The ope-sourced dataset and artefacts enables to investigate new optimization approaches and use cases by researchers.