File-Level EC : A Chunk Containing Meaning
Jun Yamada, Takaki Nakamura, Hitoshi Kamei · 2025
In recent years, with the widespread use of mobile devices such as smartphones laptop PCs etc., the amounts of data exchanged over the internet has been increasing. Erasure Coding (EC) which offers excellent storage efficiency is well-suited for protecting large amounts of data. Meanwhile, if the required number of chunks for restoration is not available, the remaining data chunks are wasted. So, we propose File-Level EC (FEC) which enables partial data recovery by mapping files to data chunks. However, it remains as challenges to implement FEC in storage software and verify practical performance. Specifically, FEC needs to be integrated into a storage software, and its overhead compared to conventional EC is quantitatively evaluated. In this paper, we describe the implementation of FEC for MinIO and evaluate its performance. We evaluated the throughput during data transfer using the MinIO client and the encoding processing time within the MinIO server. We confirmed that FEC has an overhead of within 20% compared to conventional EC, which is within a sufficiently practical range. We concluded that FEC is an effective method applicable in real-world operational environments by the evaluation.