K-nearest Neighbors Ratio Sum Discriminant Analysis Algorithm
Heqing Lu · 2023
As the dimensionality of data processed in the field of machine learning continues to increase, the issue of dimensional catastrophe often arises, necessitating the reduction of data dimensions. Traditional methods for dimensionality reduction typically construct trace ratio problems to optimize solutions based on data variance, but these problems tend to be biased towards smaller variances. The ratio sum criterion addresses this problem by transforming the trace ratio problem into a ratio sum form, thereby preventing subspaces from being projected into directions with smaller variances. However, it primarily focuses on the global variance of the data, neglecting the preservation of local characteristics. To address this limitation and preserve the local characteristics of the data, a novel algorithm called K-Nearest Neighbor Ratio Sum Discriminant Analysis (K-RSDA) is proposed. This algorithm combines k-nearest neighbor composition with the ratio sum algorithm to construct intra-class and inter-class graphs within the data. By doing so, the K-RSDA algorithm enables the ratio sum approach to effectively handle multimodal data and enhance the dimensionality reduction performance. Experiments on several small datasets in the UCI database and face datasets such as Pose27, UMIST, LFW were conducted to verify the effectiveness of the algorithm.