Storage and Compute Resource Management via DYRE, 3DcacheGrid, and CompuStore
Ioan Raicu, Ian T Foster · 2007
Both the industry and academia have an increase demand for good policies and mechanisms to efficiently manage large data sets and large pool of compute resources that ultimately perform computations on the data sets. We define large datasets to contain millions of objects and terabytes of data; these kinds of datasets are common in the Astronomy domain [1, 2, 3], Medical Imaging domain [4], and many other science domains. Large pool of compute resources could be anything from 10s to 1000s of separate physical resources geographically distributed. In order to enable the efficient dynamic analysis of large datasets, we propose three different systems that can interoperate with each other in order to offer a complete storage and resource management solution.