Measuring Generality of Documents
Hyun Woong Shin, Eduard H. Hovy, Dennis J. McLeod, Larry Pryor · 2006
Most traditional Information Retrieval (IR) systems, including web search engines, operationalize "relevant" as the word frequency in a document of a set of keywords. Because of this limitation, traditional IR systems frequently retrieve irrelevant documents in response to a user’s request. In this paper, we propose a new criterion, "generality," that provides an additional basis on which to rank retrieved documents. The generality is a level of abstraction to retrieve results based on desired generality appropriate for a user’s knowledge and interests. We compared our generality quantification algorithm with human judges’ weighting of values to show that the developed algorithm is significantly correlated.