A signal-to-noise approach to score normalization
Avi Arampatzis, Jaap Kamps · 2009
Score normalization is indispensable in distributed retrieval and fu-sion or meta-search where merging of result-lists is required. Dis-tributional approaches to score normalization with reference to rel-evance, such as binary mixture models like the normal-exponential, suffer from lack of universality and troublesome parameter estima-tion especially under sparse relevance. We develop a new approach which tackles both problems by using aggregate score distributions without reference to relevance, and is suitable for uncooperative engines. The method is based on the assumption that scores pro-duced by engines consist of a signal and a noise component which can both be approximated by submitting well-defined sets of arti-ficial queries to each engine. We evaluate in a standard distributed retrieval testbed and show that the signal-to-noise approach yields better results than other distributional methods. As a significant by-product, we investigate query-length distributions.