Syllable-Based Burrows-Wheeler Transform.
Jan Lánský, Katsiaryna Chernik, Zuzana Vlckova · DATESO · 2007
The Burrows-Wheeler Transform (BWT) is a compression method which reorders an input string into the form, which is prefer- able to another compression. Usually Move-To-Front transform and then Human coding is used to the permutated string. The original method (3) from 1994 was designed for an alphabet compression. In 2001, versions working with word and n-grams alphabet were presented. The newest version copes with the syllable alphabet (7). The goal of this article is to compare the BWT compression working with alphabet of letters, sylla- bles, words, 3-grams and 5-grams. ways of document parsing and their influence on BWT, we decided to deal with natural units of the text: letters, syllables and words; and compare these approaches with each other and also with the methods that divide the text into unnatural units - N-grams. In our measurements we found out that depending upon the language, words have 5-6 letters and syllables 2-3 letters in average. Therefore 3-grams were chosen to conform to the syllable length and 5-grams to correspond to average