Eliminating replica selection - using multiple replicas to accelerate data transfer on grids
Feng Jun, Marty Humphrey · 2004
Data-intensive, high-performance computing applications often require the efficient transfer of terabytes or even petabytes of data in wide-area, distributed computing environments. To increase the efficiency of wide area data movement, researchers have devised various techniques such as TCP tuning, multiple streams and asynchronous I/O. This paper adopts an approach to increase performance further by exploiting replica-level parallelism in grids. rFTP, a grid data transferring tool, improves the data transfer rate and reliability on grids by utilizing multiple replica sources concurrently. Experiments on the NPACI grid show as much as a 2.02/spl times/ speedup over a single data source by adaptively retrieving partial data segments from 4 replicas using the data provided by NWS.