Efficiency vs. Effectiveness in Terabyte-Scale Information Retrieval.
Stefan Büttcher, Charles L. A. Clarke · 2005
We describe indexing and retrieval techniques that are suited to perform terabyte-scale information retrieval tasks on a standard desktop PC. Starting from an Okapi-BM25-based default baseline retrieval function, we explore both sides of the effectiveness spectrum. On one side, we show how term proximity can be integrated into the scoring function in order to improve the search results. On the other side, we show how index pruning can be employed to increase retrieval efficiency -- at the cost of reduced retrieval effectiveness. We show that