Use of erasure code for low latency cloud storage
Guanfeng Liang, Ulaş C. Kozat · 2014
Recent literature including our past works provide analysis and solutions for using (i) erasure coding, (ii) parallelism, or (iii) variable slicing/chunking (i.e., dividing an object of a specific size into a variable number of smaller chunks) in speeding up the I/O performance of storage clouds. Generally, such systems can be characterized by the tuple of parameters (n, k, L): each data file is represented by k equally sized data chunks and encoded into at least n coded chunks; there are L parallel/independent servers/connections in total, each of which can download 1 coded chunk at a time; retrieval of a file is carried out by downloading n coded chunks in parallel and considered finished upon completion of any k downloading jobs. Bounds have been developed for very restricted special cases (n = k and n = L) with exponential service time, and very little is known when k <; n <; L and non-exponential service time. We investigate and develop bounds and approximations for the general case. We also present latest experimental results on one of the most popular public cloud storage service.