Supervised Feature Selection With Orthogonal Regression and Feature Weighting
Xia Wu, Xueyuan Xu, Jianhong Liu, Hailing Wang, Bin Xiang Hu, Feiping Nie · IEEE Transactions on Neural Networks and Learning Systems · 2020
Effective features can improve the performance of a model and help us understand the characteristics and underlying structure of complex data. Previously proposed feature selection methods usually cannot retain more discriminative information. To address this shortcoming, we propose a novel supervised orthogonal least square regression model with feature weighting for feature selection. The optimization problem of the objective function can be solved by employing generalized power iteration and augmented Lagrangian multiplier methods. Experimental results show that the proposed method can more effectively reduce feature dimensionality and obtain better classification results than traditional feature selection methods. The convergence of our iterative method is also proved. Consequently, the effectiveness and superiority of the proposed method are verified both theoretically and experimentally.