Efficient mining of textual associations
A. Gil, Gaël Dias · 2004
We describe an efficient implementation for mining textual associations from text corpora. In order to tackle real world applications, efficient algorithms and data structures are needed to manage, in reasonable time and space, the overgrowing volume of text data. For that purpose, we introduce a global architecture based on masks, suffix arrays and multidimensional arrays to implement the SENTA extractor (Dias, 2002). In particular, SENTA has shown great flexibility and accuracy for mining textual associations such as collocations, cognates, morphemes and chunks. Our solution shows O(h(F) N log N) time complexity and O(N) space complexity where N is the size of the corpus and h(F) is a function of the context window size.