Having your cake and eating it too: jointly optimal erasure codes for I/O, storage and network-bandwidth
K. V. Rashmi, Preetum Nakkiran, Jingyan Wang, Nihar B. Shah, Kannan Ramchandran · 2015
Erasure codes, such as Reed-Solomon (RS) codes, are increasingly being deployed as an alternative to data-replication for fault tolerance in distributed storage sys-tems. While RS codes provide significant savings in storage space, they can impose a huge burden on the I/O and network resources when reconstructing failed or otherwise unavailable data. A recent class of era-sure codes, called minimum-storage-regeneration (MSR) codes, has emerged as a superior alternative to the popu-lar RS codes, in that it minimizes network transfers dur-ing reconstruction while also being optimal with respect to storage and reliability. However, existing practical MSR codes do not address the increasingly important problem of I/O overhead incurred during reconstructions, and are, in general, inferior to RS codes in this regard. In this paper, we design erasure codes that are simultane-ously optimal in terms of I/O, storage, and network band-width. Our design builds on top of a class of powerful practical codes, called the product-matrix-MSR codes. Evaluations show that our proposed design results in a significant reduction the number of I/Os consumed dur-ing reconstructions (a 5 × reduction for typical parame-ters), while retaining optimality with respect to storage, reliability, and network bandwidth. 1