Model for planning of distributed data production
Dzmitry Makatun, Jérôme Lauret, Hana Rudová, Michal Sumbera · 2015
We propose a model of distributed data production, where input files from a single source are processed at several remote sites (each file once) and output is transferred back. The model is formulated using a network flow maximization approach and allows planning of data transfers and scheduling of CPU loads and disk storages. This class of problems can be solved in polynomial time using known algorithms. Such an approach enables automated online planning and optimization which are highly demanded in data intensive computational fields such as High Energy and Nuclear Physics.