Transform-based Lossy Compression for HPC Big Datasets
Aekyung Moon, Juyoung Park, Yun Jeong Song · 2021 International Conference on Information and Communication Technology Convergence (ICTC) · 2021
As many IoT devices generate an enormous and varied amount of data that need to be processed in a brief space of time, storing and processing IoT big data becomes a huge challenge. While lossy compression can drastically reduce data volume, finding an optimal balance between volume reduction and information loss is not an easy task. Motivated by this, we analyze the characteristics of data compressed and present relationships between compression ratio and data characteristics to use an effective lossy compression algorithms.