Contextual Features and Sequence Labeling Techniques for Relevance Prediction in Retrieval
Alberto Oliveira, Anderson De Rezende Rocha · 2020
Query performance prediction (QPP) is the name given to approaches that aim at predicting, usually post-query, the quality of results generated by an IR system. Yet, the indirect way that standard QPP approaches estimate performance is opaque, has little relation to the way users evaluate queries, and is limited in application. An alternative, more challenging way to look at the QPP problem is as a relevance prediction problem. Instead of resulting in a quality estimate, relevance prediction aims at predicting the relevance of the top-k results in a rank. When accurate, those predictions can be directly employed as a mechanism to improve the initial rank. In this work, we explore a learning framework based on sequence labeling combined with contextual features for relevance prediction. Sequence labeling is a common approach to deal with sequential data, learning from the sequence to generate a sequence of predicted labels. The context within a rank, that is, the neighborhood of ranked objects, has been successfully explored within rank-aggregation and re-ranking scenarios, and here we extend the idea to devise an effective relevance prediction feature. We show that the combination of both can beat the baseline relevance prediction approach, performing effective prediction, that is accurate for up to k = 30 positions.