Evaluating database selection techniques
James C. French, Allison L. Powell, Charles L. Viles, Travis Emmitt, Kevin J. Prey · 1998
We describe a testbed for database selection techniques and an experiment conducted using this testbed.The testbed is a decomposition of the TREC/TIPSTER data that allows analysis of the data along multiple dimensions, including collection-based and temporal-based analysis.We characterize the subcollections in this testbed in terms of number of documents, queries against which the document,s have been evaluated for relevance, and distribution of relevant documents.We then present initial results from a study conducted using this testbed that examines the effectiveness of the gGlOSS approach to database selection.The databases from our testbed were ranked using the gGl0S.S techniques and compared to the gGlOSS I&l(l) baseline and a baseline derived from TREC relevance judgements.We have examined the degree to which several gGlOSS estimate functions approximate these baselines.Our initial results confirm that the gGZOSS estimators are excellent predictors of the Ideal(Z) ranks but that the Ideal(l) ranks do not estimate relevance-based ranks well.