Ant Colony Optimization applied to the regularization process of a High Order neural Network
Ashraf M. Abdelbar, Khalid M. Salama · 2016
High order neural networks (HONN's) are neural networks that employ neurons which combine their inputs non-linearly. HONEST (High Order Network with Exponential SynapTic links) is a HONN that employs product units, and inter-neuronal connections with associated adaptable exponential weights. Previous work has found that HONEST benefits from the inclusion in the network's error function of a regularization term that penalizes high-magnitude exponents. In the present work, we use ACOℝ, a recent Ant Colony Optimization (ACO) algorithm, to optimize the parameters of HONEST's exponent regularization process, using a collection of UCI datasets. We then evaluate HONEST with the evolved parameters on a second non-overlapping collection of UCI datasets against Support Vector Machines (SVM). We find HONEST's test set predictive accuracy to be competitive with SVM, with no statistically significant difference between the two.