When Uniform Replication Meets Non-uniform Data Popularity: Efficiency Crisis and Its Solution
Xiangyu Luo · Journal of Information and Computational Science · 2014
Uniform replication is widely employed in distributed file systems to improve data reliability and I/O performance. However, non-uniform distributions of data popularity are observed in many real-world applications. The practical difference between uniform replication and non-uniform distributions of data popularity may impair the efficiency of resource utilization. To address the problem, we make the first attempt at a theoretical analysis. We draw an important conclusion that, with non-uniform data popularity, uniform replication suffers severe decline of efficiencies in the utilization of disk space and I/O bandwidth as system scales up. Moreover, the effectiveness of prospective tiered storage scheme is demonstrated in the paper. A formula is derived to determine the least number of files required to be transferred to the storage tier composed of high-end devices.