Runtime Estimation Using Linear Regression Method in Hadoop MapReduce
Nour-Eddine Bakni, Ismail Assayad · 2024
Hadoop is one of the open-source frameworks and most used by big data platforms to store and process massive amounts of data across a cluster of nodes in parallel. For Big Data clients who are generally seeking to minimize their costs, they are interested in knowing the estimated runtime for their workloads, and therefore the approximate budget. In this context, this paper proposes a method to estimate the processing time by using the linear regression technique in the Hadoop MapReduce model. The experimental results show good accuracy of this method in estimating jobs execution time.