INFORMATION RETRIEVAL TEST COLLECTIONS
Karen Spärck Jones, C. J. van Rijsbergen · Journal of Documentation · 1976
Many retrieval experiments have been based on inadequate test collections, and current research is hampered by the lack of proper collections. This short review does not attempt a fully documented survey of all the collections used in the past decade: hopefully representative examples have been studied to throw light on the requirements test collections should meet, to show how past collections have been defective, and to suggest guidelines for a future ‘ideal’ test collection. The specifications for this collection can be taken as an indirect comment on our present state of knowledge of major retrieval system variables, and experience in conducting experiments.