Evaluating Term Concept Association Mesaures for Short Text Expansion: Two Case Studies of Classification and Clustering.
Alessandro Marco Boutari, Claudio Carpineto, Daniela D’Aloisi · Concept Lattices and their Applications · 2010
The proliferation of Web applications based on short texts represents both an opportunity and a challenge to text mining algorithms, because of sparse representations and lack of shared context. To address this problem, we investigate a term expansion approach based on analyzing the relationships between the term concepts present in the concept lattice associated with the document corpus. We define five term concept association measures: proximity, concept similarity, connection strength, damping-weighted proximity, proximity&strength. By means of two case studies, we evaluate the effectiveness of these measures for expansion-enhanced K-NN classification and K-Means clustering of short texts. The results suggest that the five measures are highly competitive, with the best measure showing a clear improvement over the corresponding unenhanced K-NN and K-Means algorithms, as well as over two alternative term expansion enhancements (i.e., based on Wordnet and on pseudo-relevance feedback).