Xerox site report : Four TREC-4 tracks
Marti A. Hearst, Jan Ole Pedersen, Peter L. T. Pirolli, Hinrich Schütze, Gregory Grefenstette, David A. Hull · 1995
this document sample than one would expect by chance. The terms are selected according to a binomial likelihood ratio test [10], comparing their occurrence in the first 20 documents to their occurrence in the rest of the collection. The selected terms are then weighted in proportion to the significance of their occurrence in the sampled documents. Since it uses the baseline results, this run may also be affected by the programming error described above. query set base infl infl-np expand all Q1-25 0.454 0.484 0.492 0.467 Q26-50 0.174 0.204 0.212 0.267 20 Q1-25 0.718 0.718 0.722 0.722 Q26-50 0.306 0.354 0.378 0.402 Table 8: Average precision at all relevant docs (all) and average precision at 20 docs (20) for Spanish queries. The corrected Spanish performance figures are presented in Table 8. We include four different runs: (1) base = stop list but no morphological analysis, (2) infl = text lemmatized (stemmed) with inflectional morphology, (3) infl-np = noun phrase weight doubled, and (4) expand = query expansion. The uncorrected performance figures for infl-np and expand on Q26-50 (corresponding to our submitted runs) are 0.190/0.366 and 0.238/0.380 respectively. We present separate results for queries 1-25 (used for TREC-3) and 26-50 (used for TREC-4) since the former are substantially longer, and we note that the results reflect the difference in length. We find that lemmatization using inflectional morphology helps in most cases, making a 3-5% absolute difference in performance. However, when the queries are long and the user is examining fewer than 20 documents, there is no improvement. These conclusions agree with the results obtained for English [13], although the Spanish inflectional morphology is somewhat more effective than its English counterpart. Doubling ...