MLED: A Layered Architecture for Reducing Undetected Error Probability in File Transfer
Arash Sarabi, Ibrahim Matta, Violet R. Syrotiuk · IEEE Communications Letters · 2021
Reliability in petabyte-scale file transfers is critical for data collected from scientific instruments. We introduce a Multi-Layer Error Detection (MLED) architecture that significantly reduces the Undetected Error Probability (UEP) in file transfer. MLED is parameterized by a number of layers$n$, and a policy$P_{i}$for each layer$1 \le i \le n$that describes its operation. MLED generalizes existing error detection approaches used in file transfer. We show conditions under which MLED reduces UEP. Analytical results show that a petabyte-size file transfer in MLED with$n=2$using CRC-32s improves UEP by$2.49E\,+\,28$compared to a single-layer CRC-64, when the BER is 10−10.