Automated Hidden Neuron Optimization for Multilayer Perceptrons for Classification Tasks
Susmitha Boyidapu, Lakshmi Kavya Kalyanam, Srinivas Katkoori · 2024
Determining the optimal number of hidden neurons in a multi-layer perceptron (MLP) network remains a significant challenge in machine learning, with no universally optimal method identified to date. While various approaches such as analytical, pruning, constructive, and evolutionary methods have been proposed, they often fall short of providing an ideal solution. This study presents a novel approach to address this challenge, combining preprocessing, feature selection, and strategic MLP architecture construction. Our proposed system begins with a comprehensive data preprocessing stage, followed by a hybrid feature selection method to identify the most relevant input variables. Utilizing these selected features, we construct an MLP architecture and implement k-fold cross-validation for robust training to find the optimal number of hidden neurons. To prevent overfitting and unnecessary computational expense, we incorporate an early stopping mechanism. The key innovation of our approach lies in its ability to dynamically determine the optimal number of hidden neurons while simultaneously maximizing model performance. By iteratively adjusting the hidden layer configuration and evaluating performance metrics, our method identifies the most effective network structure for the given classification task. We compare the experimental results of seven classification datasets from the UCI machine learning library and Kaggle to evaluate our method. The breast cancer dataset achieves 95.07 % accuracy with two hidden neurons. We find the optimal network configuration that has fewer hidden neurons and has good model performance.