Variable selection for multicategory SVM via sup-norm regularization
Hao Helen Zhang, Yue-Fu Liu, Yichao Wu, Ji Zhu · NCSU Libraries Repository (North Carolina State University Libraries) · 2006
The Support Vector Machine (SVM) has been a popular classification method in machine learning and has enjoyed great successes for many applications.However, the standard SVM cannot select variables automatically and consequently its solution typically utilizes all input variables.This makes it difficult to identify important variables which are predictive of the response and can be a concern for many problems.In this paper, we propose a novel type of regularization for the multicategory SVM (MSVM), which automates the process of variable selection and results in a classifier with enhanced interpretability and improved accuracy, especially for high dimensional low sample size data.The MSVM generally requires estimation of multiple discriminating functions and applies the argmax rule for prediction.For each individual variable, we propose to characterize its importance by the supnorm of its coefficient vector associated with different functions, and then minimize the MSVM hinge loss function subject to a penalty on the sum of supnorms.The adaptive regularization, which imposes different penalties on different variables, is studied as well.Moreover, we develop an algorithm to compute the proposed supnorm MSVM effectively.Finally, the performance of the proposed method is demonstrated through simulation studies and an application to microarray gene expression data.