Erasure Codes for IPFS
Pratik Shinde · Maryland Shared Open Access Repository (USMAI Consortium) · 2019
IPFS (Interplanetary File System) is a peer to peer method of storing content addressable data in distributed file system. To provide Fault Tolerance to important files, IPFS copies them on at least 3 nodes in different geographical areas. To store a file safely, IPFS consumes at least 3X more space. This is inefficient and can be reduced significantly using Erasure Coding. My idea is to use Reed-Solomon code to compute parity of data. For each N blocks of data, Reed Solomon codes will produce additional ?K? parity blocks. Each of these ?N+K? blocks can be stored on different IPFS nodes. Original data can be reconstructed even if any K nodes are not available. To provide K-Fault tolerance to the system, we only need N+K storage as opposed to N*K in the earlier approach.