Automated Feature Engineering using Kernel Functions
Puneet Mahajan · 2020
We investigate the potential of using the various kernel functions common to Support Vector Machine(SVM) based modeling from a feature-engineering perspective. Recently, with the advent of Automated Machine Learning, significant research efforts have been placed in areas such as Neural Architecture Search, Hyper Parameter Optimization, Meta Learning, Automated Data Cleaning and Automated Feature Engineering(Auto FE). Auto FE has had some research around automated linear and nonlinear systems feature generation.We have taken a concrete approach towards feature generation taking cue from SVM's kernel tricks which intend to move features from non-linear relationships in lower dimensions to linear relationships in higher dimensions. While the approach was developed for better classification of features in a linear hyperspace, the same technique could be used to assist other classification techniques by auto generating features which add lift to existing feature space.Our approach has had positive uplift for some of the simplistic kernels within SVM literature and has had negative uplift for the more complicated kernel functions. The baseline and experimental tests were done on standard datasets available for research and both datasets were passed through a standard gradient boosting model to handle multi-collinearity.