Hyperparameter Optimization Under Shell
Ivan Gridin · Apress eBooks · 2022
In the previous chapter, we saw that simple HPO techniques could produce very impressive results. Hyperparameter Optimization not only optimizes a specific model for a dataset but can even construct new architectures. But the fact is that we have used an elementary set of tools for HPO tasks so far. Indeed, up to this point, we have only used the primitive Random Search Tuner and Grid Search Tuner. We learned from the previous chapter that search spaces could contain millions and hundreds of millions of parameters. And if we had unlimited time, we could always use the Grid Search Tuner. But unfortunately, this approach is not applicable in reality. We need Tuners that strike a good balance between speed and quality in finding the best hyperparameters. Another helpful technique is Early Stopping algorithms. Early Stopping algorithms analyze the model training process based on intermediate results and decide whether to continue training or stop it to save time. This chapter will greatly enhance the practical application of the Hyperparameter Optimization approach.