Unishox: A hybrid encoder for Short Unicode Strings
Arundale Ramanathan · The Journal of Open Source Software · 2022
Unishox is a hybrid encoding technique with which short unicode strings could be compressed using context aware pre-mapped codes and delta coding resulting in surprisingly good ratios.This article discusses a hybrid encoding method for compressing Short Unicode Strings of arbitrary lengths including Latin/English text and printable special characters.This has not been sufficiently addressed by lossless short text encoding methods so far as they have one or more of the following drawbacks:• They apply only to either English text or Unicode strings and not both • They have specific criteria based on which best compression can be achieved • They are not suitable for low RAM devices To the extent we know, the existing methods available for short string compression are Smaz (Sanfilippo, 2012), Shoco (Schramm, 2015), SCSU, (A Standard Compression Scheme for Unicode -UTR #6, 2005), BOCU (Scherer & Davis, 2002), SSE (Juncai Xu et.al, 2017) and AIMCS (Abedi & Pourkiani, 2020).