Knowledge structuring for database mining and text retrieval using past optimal queries
Hayri Sever · Medical Entomology and Zoology · 1995
This dissertation examines issues of knowledge structuring in rough set theory in the context of database mining, and reusing past optimal queries in Information Retrieval (IR). The rough set methodology is extended to handle some problems of exploring very large databases that can be attributed to the data being redundant, incomplete, noisy, and dynamic. We present a Stepwise Backward Selection (SBS) for removing superfluous features, which is based on the monotonicity of classification quality. We prove that the SBS algorithm finds a small subset of relevant features in polynomial time and that they are sufficient and necessary to define target concepts with respect to a given threshold. We propose an elementary classification method in an algebraic approximation space such that it is suitable for noisy and incomplete data. Elementary classifiers are, however, unable to handle dynamic and incomplete data properly. We exploit the inconsistency property of upper classification methods in order to keep their decision algorithms from becoming obsolete. We show that upper classifiers could be just as well interpreted as if they were elementary classifiers. A number of techniques are used in IR systems to exploit user feedback in order that the system can improve its performance with respect to a particular information need. This process involves the formulation of an optimal query that best separates the documents known to be relevant from those that are not. Since obtaining an optimal query is an expensive process, the need for mechanisms to save and reuse past optimal queries, for processing future queries, is obvious. We propose the use of a query base, a set of persistent past optimal queries, and the identification of which requires the investigation of similarity measures between queries. The query base can be used either to answer user queries or to formulate optimal queries. We justify the former case analytically and the latter case by experiment. Incorporating a query base into IR system requires the choice of similarity measures between queries. We propose three similarity measures between queries depending on the structure of a query base.