Efficient Query Evaluation on Large Textual Collections in a Peer-to-Peer Environment
Jiangong Zhang, Torsten Suel · 2005
The authors studied the problem of evaluating ranked (top-k) queries on textual collections ranging from multiple gigabytes to terabytes in size. The authors focused on the case of a global index organization in a highly distributed environment, and consider a class of ranking functions that includes common variants of the Cosine and Okapi measures. The main bottleneck in such a scenario is the amount of communication required during query evaluation. Several efficient query evaluation schemes were proposed and their performances were evaluated. The results on real search engine query traces and over 120 million Web pages show that after careful optimization such queries can be evaluated at a reasonable cost, while challenges remain for even larger collections and more general classes of ranking functions.