A Cost Efficient Framework for Managing Distributed Resources in a Cluster Environment
Amril Nazir, Hao Liu, Søren-Aksel Sørensen · 2009
Most resource management systems in large distributed environments rely on a centralised controller scheme, known as a meta-scheduler. Such a meta-scheduler is designed to have a complete control of all resources worldwide through a number of local schedulers on each participating site. Faced with a potential large number of sites, the central scheduling point represents a potential bottleneck for requests needing fast response. In this paper, we consider the efficiency of a localised two-tier system: the first tier selects appropriate computing resources from the worldwide pool and creates a virtual cluster. The second tier employs a local scheduler to distribute jobs/tasks from the applications among the resources that make up the cluster. The two-tier approach differentiates the tasks of securing a suitable resource level and managing their use. The aim is to ensure a rapid request response while at the same time optimising the system performance. To this end, we consider a management strategy that incorporates rental policy and scheduling schemes. We propose aggressive and conservative heuristic based policies and evaluate their performance to demonstrate the cost effectiveness of such approaches.