Feature extraction through local learning
Yijun Sun, Dapeng Oliver Wu · Statistical Analysis and Data Mining The ASA Data Science Journal · 2009
Abstract RELIEF is considered one of the most successful algorithms for assessing the quality of features. It has been recently proved that RELIEF is an online learning algorithm that solves a convex optimization problem with a margin‐based objective function. Starting from this mathematical interpretation, we propose a novel feature extraction algorithm, referred to as local feature extraction (LFE), as a natural generalization of RELIEF. LFE collects discriminant information through local learning and can be solved as an eigenvalue decomposition problem with a closed‐form solution. A fast implementation of LFE is derived. Compared to principal component analysis, LFE also has a clear physical meaning and can be implemented easily with a comparable computational cost. Compared to other feature extraction algorithms, LFE has an explicit mechanism to remove irrelevant features. Experiments on synthetic and real‐world data are presented. The results demonstrate the effectiveness of the proposed algorithm. © 2009 Wiley Periodicals, Inc. Statistical Analysis and Data Mining 2: 34‐47, 2009