Resource Prediction for Big Data Processing in a Cloud Data Center : A Machine Learning Approach
Alanazi Rayan, Yunmook Nah · IEIE Transactions on Smart Processing and Computing · 2018
The high demand for big data applications, such as the Internet of Things (IoT), healthcare, business, and academia, as well as government, fosters the creation of large-scale cloud data centers. Cloud data centers contain thousands of physical machines (PMs), so resource management is necessary for allocating the tremendous amount of data to them. Knowing the workload demand in advance enables control of those resources, saving energy, reducing CPU and memory usage, and improving service. Workload prediction can be used to determine how many resources need to be allocated in the future. In this paper, we propose machine learning–based techniques to predict the daily operational workload. The proposed approach can predict the amount of power consumption (PC) and the number of PMs required to fulfill the demands of the cloud data center. Workload prediction accuracy varies based on the prediction methods used and the type of workload. In this work, we investigate three different methods: polynomial regression, support vector regression, and random forest regression (RFR). Considering both accuracy and computation time, results show that RFR provides the best performance, in our case, with a minimum root-mean-square error of 11.68 for PMs and 4869.08 for PC prediction. The computation time solidifies our selection with 2 seconds training time in all instances.