Unified Data Management System for Large Scale Task Parallel Computing
Lian Li · Journal of Chinese Computer Systems · 2011
Large scale task parallel computing usually involves a large amount of data that can be individually scheduled on different computing resources.With the development of the grid technology in scientific computing,the dataset of large scale task parallel computing has increased more and more rapidly.And scientists usually divide the large scale dataset into small groups and upload them to different grid platforms to reduce the time for the whole task.This paper proposes a unified data management system for large scale task parallel computing which logically integrates the data space in heterogeneous grid platforms.The system also provides a transparent interface for all heterogeneous data spaces.And application of the system to large scale virtual screening shows that the system has the high efficiency of data transfer and management.