Closed-form supervised dimensionality reduction with generalized linear models
Irina Rish, Genady Ya. Grabarnik, Guillermo Cecchi, Francisco Pereira, Geoffrey J. Gordon · 2008
We propose a family of supervised dimensionality reduction (SDR) algorithms that combine feature extraction (dimensionality reduction) with learning a predictive model in a unified optimization framework, using data- and class-appropriate generalized linear models (GLMs), and handling both classification and regression problems. Our approach uses simple closed-form update rules and is provably convergent. Promising empirical results are demonstrated on a variety of high-dimensional datasets.