Statistical and graph-based approaches to small sample and high dimensional data
Ikumi Suzuki · Institutional Repositories DataBase (IRDB) · 2012
In recent years, machine learning has become a popular tool for analyzing various types of data.The goal of machine learning is to construct a model from existing data to make predictions for new data.To build (or select) accurate predictive models, two aspects of data must be verified: (1) Is the amount of training data large enough to predict unseen test data?(2) When the data is represented by a vector, is the number of dimensions small enough not to be affected by so-called "curse of dimensionality"?If any of these is violated, learning a predictive model becomes much harder, but these are not necessarily satisfied in real situations.This thesis deals with how to alleviate the problems incurred in such situations.For issue (1), we address the task of selecting a predictive model using a small number of training samples.In particular, we focus on developing a cancer diagnosis system that requires an accurate prediction from gene expression profiling (microarray) data.We propose a "min-max" model selection method based on the bootstrap resampling to obtain a reliable classifier.We show that our method is less susceptible to variation in the assessment of the occurrence data, indicating the effectiveness of risk-averse as a model selection criterion.For issue (2), we focus on a problem related to the high dimensionality of the data called hubness phenomenon, which was discovered only recently.We show the family of kernels based on the graph Laplacian is less prone to make hubs when used as a similarity measure.We found that these kernels indeed reduce hubness phenomemon in some cases, and in these cases they work well in ranking and classification tasks.This result suggests that the amount of hubs, which can be readily computed in an unsupervised fashion, can be a yardstick of whether Laplacian-based kernels work effectively for a given data.