Latent supervised learning and DiProPerm
Susan H. Wei · Carolina Digital Repository (University of North Carolina at Chapel Hill) · 2019
The field of machine learning has grown rapidly in recent decades with a diverse range of applications. This dissertation contributes novel machine learning techniques motivated by modern biomedical challenges where data is often characterized by high dimensionality. Personalized medicine serves as the motivating application for the first methodology introduced. One of the underlying premises of personalized medicine is that effectiveness of specific treatments may be heterogeneous across people. We develop a new machine learning task called Latent Supervised Learning that can, among other tasks, estimate treatment effect heterogeneity. More broadly, Latent Supervised Learning is designed for data settings that do not fall under the traditional frameworks provided by supervised and unsupervised learning, the two most common categories of machine learning. High dimensional low sample size data settings motivate the development of the second methodology presented in this dissertation. Direction-Projection-Permutation (DiProPerm) is a general nonparametric framework for testing equality of distributions. The idea is to project high-dimensional data onto a direction that discriminates between the two populations and to infer differences in the higher dimensional distributions from their lower dimensional projections. This use of lower dimensional projections makes DiProPerm a natural companion to high dimensional data visualization. Theoretical properties of DiProPerm are investigated under a non-classical asymptotic regime which reveals surprising properties of hypothesis tests in high dimensional low sample size settings.