Hybrid Dynamic Scheduling of MapReduce and Spark Services Based on the Profit Model in the Cloud Computing Platform
Jing Hu · 2021
The big data technology has been efficient in carrying out analysis and processing of massive data. Many service providers build cloud platforms based on big data technology to provide users with computational requirements. The goal of this paper is to, targeting users who are uncertain about service deadlines and who have urgent needs to complete their services, develop an efficient, two-type, offline service scheduling strategy that enables the service provider to achieve maximum profit, at the same time determines relatively exact deadlines for the user, and empowers the resource of the cloud platform to reach optimum utilization rate, thus realizing mutual benefit between service providers and users. In this paper, we first propose a Profit Model with reward and penalty, as a criterion of profit maximization. This paper then put forward the MapReduce and Spark services hybrid scheduler (MRSHS) targeting Profit Model, which contains the determining Round Number (MRN) algorithm facing MapReduce service as well as the Hybrid Dynamic Scheduling (HDS) algorithm facing MapReduce and Spark services. Finally, a large number of experiments prove that, compared with existing schedulers, MRSHS put forward in this paper reaches a fairly high level of efficiency and availability in terms of service performance, the profit for service providers, and platform resource utilization.