Linear dimensionality reduction for multi-label classification
Shuiwang Ji, Jieping Ye · 2009
Dimensionality reduction is an essential step in high-dimensional data analysis. Many dimension-ality reduction algorithms have been applied suc-cessfully to multi-class and multi-label problems. They are commonly applied as a separate data pre-processing step before classification algorithms. In this paper, we study a joint learning framework in which we perform dimensionality reduction and multi-label classification simultaneously. We show that when the least squares loss is used in classifi-cation, this joint learning decouples into two sepa-rate components, i.e., dimensionality reduction fol-lowed by multi-label classification. This analysis partially justifies the current practice of a separate application of dimensionality reduction for classi-fication problems. We extend our analysis using other loss functions, including the hinge loss and the squared hinge loss. We further extend the for-mulation to the more general case where the in-put data for different class labels may differ, over-coming the limitation of traditional dimensionality reduction algorithms. Experiments on benchmark data sets have been conducted to evaluate the pro-posed joint formulations. 1