Attention Feature Selection Based on Prior Knowledge Injection
Haifeng Zhao, Quanman Chen, Lili Huang, Yanping Fu · 2023
High-dimensional data generally appear in various analysis tasks, but high-dimensional data often causes computational waste and noise interference. Feature selection can make the model achieve better performance by preserving the optimal feature subset in the data to solve the problem of data dimensionality reduction. It is a popular method to select feature subset by combining autoencoder structure reconstruction. However, most autoencoder methods only take reconstruction as the objective function, which will lead to a decrease in the ability of feature discrimination for data with obvious clustering structure. Moreover, only decoder is used as an auxiliary structure, and the performance of the decode cannot be further released, resulting in the sub-optimal feature subset obtained. In this paper, we obtain the internal clustering matrix of the data in advance through manifold learning and spectral analysis, which we call prior knowledge. By constraining the distance between the decoder output and the prior knowledge, the model can force learning the clustering information of the data and improve the ability to identify the feature. Secondly, an attention module is designed to highlight the feature information of key locations. We prove the effectiveness of our method by experiments on several standard datasets.