FastReplica: efficient large file distribution within content delivery networks
Ludmila A. Cherkasova, Jang-Won Lee · 2003
Abstract. In this work, we consider a large-scale dis-tributed network of servers and a problem of content dis-tribution across it. We propose a novel algorithm, called FastReplica, for an efficient and reliable replication of large files in the Internet environment. There are a few basic ideas exploited in FastReplica. In order to replicate a large file among n nodes (n is in the range of 10-30 nodes), the orig-inal file is partitioned into n subfiles of equal size and each subfile is transferred to a different node in the group. After that, each node propagates its subfile to the remaining nodes in the group. Thus instead of the typical replication of an en-tire file to n nodes by using n Internet paths, connecting the original node to the replication group, FastReplica exploits n×n Internet paths within the replication group where each path is used for transferring 1 n-th of the file. We design a scalable and reliable FastReplica algorithm which can be used for replication of large files to a large group of nodes. The new method is simple and inexpensive. It does not require any changes or modifications to the existing Internet infras-tructure, and at the same time, it significantly reduces the file replication time as we demonstrate through experiments on a prototype implementation of FastReplica in a wide-area testbed. 1