Comparison of Hyperparameter Tuning Techniques on KNN Algorithm to find the Best K Value using Grid Search and Random Search Methods
Akmal Taufiq, Sri Yulianti, Alam Rahmatulloh, Irfan Darmawan, Randi Rizal · 2024
K-Nearest Neighbors (KNN) is a widely used classification algorithm, but its performance heavily depends on the value of the K parameter. Hyperparameter tuning methods like grid search and random search are employed to find the optimal K value. However, it remains unclear which method is superior due to their differing characteristics. This study compares these methods by evaluating accuracy, memory usage, and computing time. Grid Search was found to be superior in accuracy, with an average difference of 0.5% in accuracy, 0.67% in precision, 0.83% in recall, and 0.33% in F1-score. Conversely, Random Search excels in memory usage and computing time, with a difference of 3.4 MiB in memory usage and eight seconds in computing time. This research aims to guide machine learning practitioners in selecting the appropriate hyperparameter tuning method for the KNN algorithm based on specific priorities and needs.