Dynamic function placement for data-intensive cluster computing
Khalil Amiri, David Petrou, Gregory R. Ganger, Garth A. Gibson · 2000
Optimally partitioning application andfilesystem functionality within a cluster of clients and servers is a difficult problem dueto dynamic variations in application behavior, resource availability, and workload mixes. Thispaper presents A BACUS, a run-time systemthat monitors and dynamically changes function placement for applications that manipulate largedata sets. Several examples of data-intensive workloads are used to show the importance ofproper function placement and its dependence on dynamic run-time characteristics, withperformance differences frequently reaching 2-10X. We evaluate how well the ABACUSprototype adapts to run-time system behavior, including both long-term variation (e.g., filterselectivity) and short-term variation (e.g., multi-phase applications and inter-applicationresource contention). Our experiments with ABACUS indicate that it is possible to adapt inall of these situations and that the adaptation converges most quickly in those cases where theperformance impact is most significant. 1