Kernel-Based Optimal Subspaces (KOS): A Method for Data Classification
Lakhdar Remaki · Machine Learning and Knowledge Extraction · 2026
Support Vector Machine (SVM) is a popular kernel-based method for data classification that has demonstrated high efficiency across a wide range of practical applications. However, SVM suffers from several limitations, including the potential failure of the optimization process, especially in high-dimensional spaces; the inherently high computational cost; the lack of a systematic approach to multi-class classification; difficulties in handling imbalanced classes; and the prohibitive cost of real-time or dynamic classification. This paper proposes an alternative method, referred to as Kernel-based Optimal Subspaces (KOS), which belongs to the family of kernel subspace methods. Mathematically similar to Kernel PCA (KPCA), KOS achieves performance comparable to SVM while addressing the aforementioned weaknesses. The method is based on computing the minimum distance to optimal feature subspaces of the mapped data. Because no optimization process is required, KOS is robust, fast, and easy to implement. The optimal subspaces are constructed independently, enabling high parallelizability and making the approach well-suited for dynamic classification and real-time applications. Furthermore, the issue of imbalanced classes is naturally handled by subdividing large classes into smaller sub-classes, thereby creating appropriately sized sub-subspaces within the feature space.