Pongo: Efficient Lossless Floating Point Compression
Yufeng Liu, Yao Shen, Fenghua Zhang, Fei Xue Huang · Cloud Computing and Data Science · 2025
A large amount of time series data is increasingly being collected in different fields. In order to make good use of this large amount of time series data, it is necessary to solve the problems of high storage costs and transmission bandwidth that the data bring. The general compression algorithms effectively reduce the size of data at the cost of a large amount of computation. However, due to the huge time cost and batch processing mode of the general compression algorithms, Time Series Management Systems (TSMSs) often use streaming compression algorithms to replace general compression algorithms for compressing time series data. For floating-point data, most prevalent streaming compression algorithms, such as those based on exclusive OR (XOR) operations, offer relatively fast processing and high compression ratios compared to conventional generalpurpose compression algorithms. Among them, the Elf algorithm proposes the idea of first erasing and then compressing, achieving the best compression ratio among existing streaming compression algorithms. This paper proposes a new lossless streaming compression algorithm Pongo for floating-point numbers, which uses a carefully designed erasing method different from Elf. The Pongo algorithm employs a novel erasing technique that transforms the binary representation of fractional parts to decimal, leveraging a newly proposed algorithm that enhances the efficiency of this conversion process. To demonstrate the superior performance of Pongo, we conducted extensive experiments comparing it with ten leading compression algorithms across twenty-two different datasets. On average, Pongo achieves a compression ratio that is 14% better than Elf and 58% better than Gorilla, making it the top-performing algorithm among all those tested, as shown through both mathematical analysis and practical testing.