Diverse retrieval via greedy optimization of expected 1-call@k in a latent subtopic relevance model
Scott Sanner, Shengbo Guo, Thore Graepel, Sadegh Kharazmi, Sarvnaz Karimi · 2011
It has been previously observed that optimization of the [email protected] relevance objective (i.e., a set-based objective that is 1 if at least one document is relevant, otherwise 0) empirically correlates with diverse retrieval. In this paper, we proceed one step further and show theoretically that greedily optimizing expected [email protected] w.r.t. a latent subtopic model of binary relevance leads to a diverse retrieval algorithm sharing many features of existing diversification approaches. This new result is complementary to a variety of diverse retrieval algorithms derived from alternate rank-based relevance criteria such as average precision and reciprocal rank. As such, the derivation presented here for expected [email protected] provides a novel theoretical perspective on the emergence of diversity via a latent subtopic model of relevance --- an idea underlying both ambiguous and faceted subtopic retrieval that have been used to motivate diverse retrieval.