Evaluating Diversity and Redundancy-Based Search Metrics Independently
Ake Tangsomboon, Teerapong Leelanupab · 2014
This paper proposes a new evaluation metric, normalized Coverage Frequency (nCF), which aims to explicitly evaluate the diversity of search results, going beyond the drawbacks of previously proposed measures. In fact, two of the most widely adopted metrics for the diversity retrieval task, namely α-nDCG and Intent-Aware Expected Reciprocal Rank (ERR-IA), explicitly evaluate redundancy, but not diversity. While there exists a genuine diversity-based metric called Intent Recall (I-rec), it has some drawbacks. These drawbacks may be inherited by other derived metrics such as D#-nDCG, which combines I-rec with a modified version of nDCG.