An Effective Term-Weighting Scheme for Information Retrieval
YunJae Jung, Haesun Park, Ding‐Zhu Du · University of Minnesota Digital Conservancy (University of Minnesota) · 2000
The retrieval performance of the information retrieval systems is largely dependent on similarity measures. Furthermore, a term-weighting scheme plays an important role for the similarity measure. In this paper, we investigate existing weighting approaches and similarity measures to propose a new term-weighting scheme supporting the cosine similarity measure to retrieve relevant documents effectively on the basis of vector space model. The new weighting technique considers not only occurrence terms but also absence terms in finding similarity among document vector representations. The consideration of negatively weighted terms resolves the masking by zero problem of the inner product operation. According to the experimental results, the balanced term-weighting scheme performs more effectively especially when the vector space of data collection is relatively dense. Consequently, the balanced weighting scheme promises significant contribution to the relevance effectiveness for the information retrieval systems.