A Selectivity Model for Fragmented Relations in Information Retrieval
Henk Ernst Blok, R.S. Choenni, Henk Blanken, Peter M. G. Apers, Henk Ernst, Blok Sunil Choenni · University of Twente Research Information · 2001
New application domains cause todays database sizes to grow rapidly, posing great demands on technology. Data fragmentation facilitates techniques (like distribution, parallelization, and main-memory computing) meeting these demands. Also, fragmentation might help improving e#cient processing of query types such as top N. Database design and query optimization require a good notion of the costs resulting from a certain fragmentation. Our mathematically derived selectivity model facilitates this. Once its two parameters have been computed based on the fragmentation, after each (though usually infrequent) update, our model can forget the data distribution, resulting in fast and quite good selectivity estimation. We show experimental verification for Zipfian distributed IR databases. Keywords: selectivity, fragmentation, Zipf, information retrieval, databases 1 Introduction E#cient and e#ective processing of large amounts of data is of crucial importance in most computer applications,...