Optimizing SVM Performance through Combinatorial Hyperparameter Tuning and Model Selection
Hassan Tariq, Mehwish Majeed, Muhammad Imran Ahmad · International Journal Bioautomation · 2025
Among the support vector machine (SVM) methods, the support vector classifier (SVC) is widely utilized for binary and multi classification tasks across various datasets. Hyperparameter tuning plays a critical role in optimizing the performance of SVM by helping to prevent overfitting or underfitting, enhancing model stability, adapting the model to different types of datasets, and increasing predictive power. This study aims to maximize SVM performance on datasets related to heart disease, liver disorder, breast cancer, and MINST (a digit dataset), which exhibit diverse sample and feature counts. Our proposed framework leverages Python libraries. It employs a combinatorial approach to tune the kernel, C, and degree hyperparameters for both the train-test-split and cross validation (CV) models with different input values. Model accuracy, the area under the curve (AUC), and the F1 score were used to evaluate the models. The most suitable model, hyperparameters, and validation size or number of folds, are selected to achieve maximum accuracy of SVM across all datasets. Results demonstrate that the train-test-split model generally improves SVM performance, except for the heart disease dataset, on which the CV model performs well. Our contribution lies in the development of a framework that combines combinatorial hyperparameter tuning and model selection, aiming to optimize SVM performance and predictive capabilities. Future research can focus on enhancing SVM performance for large-scale datasets and exploring ensemble techniques or deep learning models to enhance its applications in real-world scenarios.