Design, Implementation and Evaluation of a Methodology for Utilizing Sources of Evidence in Relevance Feedback
Emanuele Di Buccio · Padua Research Archive (University of Padova) · 2011
The objective of an Information Retrieval system is to support the user when he searches for information by predicting the documents relevant to his information need. Prediction is performed on the basis of evidence available during the search process. User interactions are examples of sources from which this evidence can be gathered. This thesis addresses the problem of uniformly modeling heterogeneous forms of user interaction that are selected as sources for feedback. The problem of uniform source modeling is addressed by way of a complete methodology. The methodology aims at designing, implementing and evaluating a system that validates an experimental hypothesis. The hypothesis being validated regards the possible factors that can explain the user perception of relevance through the evidence gathered from the user interaction. The objective is to obtain and exploit a usable representation of the factors in the role of a new dimension of the information need representation. The methodology aims at being general and not tailored to a specific source. The methodology defines the set of steps needed for obtaining a vector subspace-based representation of the information need dimensions to further exploit this representation for relevance prediction purposes. The set of steps identified are source selection, evidence collection, dimension modeling, document modeling and prediction. This thesis shows how the methodology can be used for modeling two sources of evidence: term relationship in documents judged as relevant and the relationship between interaction features gathered from the behavior of the user when interacting with a set of documents. As for the term relationship dimension, this thesis shows that the current implementation of term relationship is feasible with a very large text collection delivered within the 2009 and 2010 Relevance Feedback tracks of the Text Retrieval Conference initiative. The methodology has supported the evaluation of term relationship for document re-ranking. As for interaction feature relationships, this thesis investigates the adoption of the user behavior dimension for document re-ranking both without query expansion and with query expansion.