Rotated Grid Search for Hyperparameter Optimization
International Journal of Machine Learning and Computing · 2022
This paper proposes a new hyperparameter search method involving elliptical grid transformations and rotations of a grid of probe points.This technique is termed "rotated grid search".We begin by motivating the method by discussing the limitations of random search.A new formalism for more efficiently probing a hyperparameter search space is then proposed.Next, we build a theoretical framework to compare hyperparameter optimization performance of rotated grid search against random search.We then evaluate both search methods empirically to quantify the marginal benefit of using one over the other.Monte-Carlo simulations on various synthetic objective functions show that rotated grid search outperforms random search over the full range of anisotropy explored in this study.Finally, we conduct a case study on a real dataset, rectangles-images, and show that rotated grid search outperforms random search in a high dimensional space.