A Distributed Task Scheduling System Suitable for Massive Environments
Chaoyu Wu, E Haihong, Meina Song · 2020
In a million-scale host-scale environment, the resource scheduling/management system becomes more and more important. How to improve resource utilization and reduce the cost of infrastructure while ensuring the performance of business operations in the data center is the core issue we must consider. And because of the complex relationship between tasks, the corresponding DAG (DAG: Directed Acyclic Graph) is often formed, making the distribution and management of tasks and resources more difficult. In order to solve the problems of tedious task scheduling and low resource utilization in a large-scale host machine environment, this article describes a general task and machine model based on the actual situation of various companies, and based on this model, implements a model suitable for massive data task scheduling scheme.