A hypergraph partitioning based approach for scheduling of tasks with batch-shared I/O
Gaurav Khanna, Nagavijayalakshmi Vydyanathan, Tahsin Kurç, Ümit V. Çatalyürek, Pete Wyckoff, Joel Saltz, Ponnuswamy Sadayappan · 2005
This paper proposes a novel, hypergraph partitioning based strategy to schedule multiple data analysis tasks with batch-shared I/O behavior. This strategy formulates the sharing of files among tasks as a hypergraph to minimize the I/O overheads due to transferring of the same set of files multiple times and employs a dynamic scheme for file transfers to reduce contention on the storage system. We experimentally evaluate the proposed approach using application emulators from two application domains; analysis of remotely-sensed data and biomedical imaging.