Job Scheduling In Cluster Computing: A Student Project
Hassan Rajaei, Mohammad B. Dadfar · 2020
Cluster computing has become widespread by utilizing COTS (Commercial-Off-The-Shelf) PCs, a high-speed network, and Linux operating system.This simple configuration of multiprocessor system can provide an excellent environment for student projects in courses such as Operating Systems, Data Communication, Distributed Programming, just to name a few.In this paper we describe an ongoing project focused on job scheduling for a cluster of processors.Job scheduling on distributed-memory parallel systems has always been a challenge.Traditional measurement factors such as job length to allocate the requested resources does not suffice.Other factors such as communication delays and synchronization overhead which are normally in the user domain, could turn out as key issues for multiprocessors' utilization.As a result, utilization of each processor in a distributed-memory parallel system may end up comparatively lower than a single processor system.Consequently, performance of the entire system may degenerate and user jobs risk waiting long in the queue before getting the requested number of processors.This project is divided into several phases.In each phase, one to three students investigate how to minimize waiting time of the jobs in the queue while allowing other projects have a scheduling policy that suits their experiment and research.This paper reports the results of the first group which focuses on variable partitioning scheme.When resources for the highest priority job are not available, then the lower priority jobs are allowed to acquire the available resources.This paper investigates and evaluates variable partitioning schemes for job scheduling on distributed-memory parallel systems.