Redundant Parallel File Transfer with Anticipative Adjustment Mechanism in Data Grids
Chao‐Tung Yang, Yao‐Chun Chi, Chun-Pin Fu · 2007
More and more applications emphasize analysis huge data and depend on the data transmission. Data Grids enable the selection, sharing, and connection of a wide variety of geographically distributed computational and storage resources for content the large-scale data-intensive application needs. Data grids consist of scattered computing and storage resources located in different countries/regions yet accessible to users. The co-allocation architecture was developed to enable the parallel download of datasets/servers from selected replica servers, and the bandwidth performance is the main factor that affects the internet transfer between the client and the server. Therefore, it is important to reduce the difference of finished time among replica servers, and manage changeful network performance during the term of transferring as well. In this paper, we proposed Anticipative Recursive-Adjustment Co-Allocation schemes, to adjust the workload of each selected replica server, which handles unwarned variant network performances of the selected replica servers. The algorithm is based on the previous assigned transfer size finished rate, to anticipate that bandwidth status on next section for adjusting the workload, and further, to reduce file transfer time in a grid environment. Our approach is usefully in instable gird environment, which reduces the wasted idle time for waiting the slowest server and decreases file transfer completion time.