When information retrieval measures agree about the relative quality of document rankings
Robert M. Losee · Journal of the American Society for Information Science · 2000
The variety of performance measures available for information retrieval systems, search engines, and network filtering agents can be confusing to both practitioners and scholars. Most discussions about these measures address their theoretical foundations and the characteristics of a measure that make it desirable for a particular application. In this work, we consider how preferences exhibited by measures of performance at two points in a search may be formally compared. Criteria are developed that allow one to determine the percent of time or conditions under which two different performance measures suggest that one document ordering is superior to another ordering, or when the two measures disagree about the relative value of document orderings. As an example, graphs provide illustrations of the relationships between precision and The author wishes to thank Dr. Robert Burgin of the North Carolina State Library for a discussion about Burgin (1999) that resulted in ideas leading to this article, as well as two anonyous referees for their suggestions.