An investigation of relevance feedback using adaptive linear and probabilistic models
R. G. Summer, William M Shaw · Text REtrieval Conference · 1996
The SMART system (v.11.0) was used as a front-end to a two-stage retrieval process. In the first stage (WSJ) documents and the description field of (ad hoc) topics were indexed by the stems of single terms; lnc and ltc weights were computed for word stems in documents and queries, respectively; and documents were ranked according to the cosine similarity of document and query vectors. Related by the initial query vector, the first 5000 documents in the ranked list for each topic constituted a condensed database for that topic. Preliminary experiments with TREC-4 topics and official relevance evaluations suggested each such database would include a high fraction of relevant documents for the associated topic, and the result was confirmed by TREC-5 results. In the second stage, initial query vectors were automaticaly refined by two relevance feedback strategies applied to the condensed database. One of us employed the adaptive linear model (uncis1), and the other used a variation of the classic probabilistic model (uncis2); relevance judgements were made independently. In uncis1, the query at a given search iteration is expanded by all terms in relevant, retrieved documents and all terms in selected, nonrelevant, retrieved documents, and documents are ranked by the inner product of document and query vectors. In uncis2, the query is expanded by all terms in relevant, retrieved documents, and documents are ranked by the cosine similarity of document and query vectors. For uncis1 and uncis2, respectively, average non-interpolated precision values over all relevant documents are 0.25 and 0.20, and average R-precision values are 0.25 and 0.21. Results show that independent relevance jugments made in uncis1 and uncis2 are quite different and have a strong effect on retrieval outcomes; our relevance evaluations also differ significantly from official relevance judgments. Retrieval performance improves when official relevance judgments are utilized by both models. For the 31 topics in which there was an official relevant document in the top 34 of the initial ranking, average non-interpolated precision values are 0.90 for the adaptive linear model and 0.59 for the probabilistic model