Handling Weighted Sequences Employing Inverted Files and Suffix Trees
Klev Diamanti, Andreas Kanavos, Christos H. Makris, Thodoris Tokis · 2014
In this paper, we address the problem of handling weighted sequences. This is by taking advantage of the inverted files machinery and targeting text processing applications, where the involved documents cannot be separated into words (such as texts representing biological sequences) or word separation is difficult and involves extra linguistic knowledge (texts in Asian languages). Besides providing a handling of weighted sequences using n-grams, we also provide a study of constructing space efficient n-gram inverted indexes. The proposed techniques combine classic straightforward n-gram indexing, with the recently proposed twolevel n-gram inverted file technique. The final outcomes are new data structures for n-gram indexing, which perform better in terms of space consumption than the existing ones. Our experimental results are encouraging and depict that these techniques can surely handle n-gram indexes more space efficiently than already existing