Knowledge discovery from labeled and unlabeled data
Tao Li, Mitsunori Ogihara · 2004
Knowledge discovery, also known as data mining, is the process of automatic extraction of novel, useful and understandable patterns/models from large datasets. Data are routinely collected and usually have different characteristics for different applications. As a result, different techniques are required for different types of data. This thesis focuses on the development of efficient techniques for learning from various types of data and on techniques for combining multiple data types. In particular, four key problems—Classification , Clustering, Semi-supervised Learning and Temporal Pattern Discovery—are studied. For classification, we propose a simple and efficient multi-class classification approach via generalized discriminant analysis and investigate the methods for automatically generating hierarchical structures to facilitate classification. For clustering, we develop a new clustering algorithm which explicitly models the subspace structure associated with each cluster, examine the entropy-based criterion in categorical clustering, and present the solutions for combining multiple clusterings. For semi-supervised learning, we provide a theoretical analysis as to why minimizing the disagreement between individual models could lead to the performance improvement in learning from multiple information sources and present a co-updating approach that attempts to minimize this disagreement using both labeled and unlabeled data. For temporal pattern discovery, we introduce algorithms for discovering temporal patterns without predefined time windows by formulating the problem as comparing two probability distributions of inter-arrival times. Extensive experiments have been conducted for all four problems, showing the effectiveness and efficacy of our approaches.