Fast classification rates for high-dimensional Gaussian generative models
Tianyang Li, Adarsh Prasad, Pradeep Ravikumar · 2015
We consider the problem of binary classification when the covariates conditioned on the each of the response values follow multivariate Gaussian distributions. We focus on the setting where the covariance matrices for the two conditional dis-tributions are the same. The corresponding generative model classifier, derived via the Bayes rule, also called Linear Discriminant Analysis, has been shown to behave poorly in high-dimensional settings. We present a novel analysis of the classification error of any linear discriminant approach given conditional Gaussian models. This allows us to compare the generative model classifier, other recently proposed discriminative approaches that directly learn the discriminant function, and then finally logistic regression which is another classical discriminative model classifier. As we show, under a natural sparsity assumption, and letting s denote the sparsity of the Bayes classifier, p the number of covariates, and n the num-ber of samples, the simple (`1-regularized) logistic regression classifier achieves the fast misclassification error rates of O s log p