A bilevel optimization approach to automated parameter tuning
Ankur Sinha, Pekka Malo, Peng Xu, Kalyanmoy Deb · 2014
Many of the modern optimization algorithms contain a number of parameters that require tuning before the algorithm can be applied to a particular class of optimization problems. A proper choice of parameters may have a substantial effect on the accuracy and efficiency of the algorithm. Until recently, parameter tuning has mostly been performed using brute force strategies, such as grid search and random search. Guesses and insights about the algorithm are also used to find suitable parameters or suggest strategies to adjust them. More recent trends include the use of meta-optimization techniques. Most of these approaches are computationally expensive and do not scale when the number of parameters increases. In this paper, we propose that the parameter tuning problem is inherently a bilevel programming problem. Based on this insight, we introduce an evolutionary bilevel algorithm for parameter tuning. A few commonly used optimization algorithms (Differential Evolution and Nelder-Mead) have been chosen as test cases, whose parameters are tuned on a number of standard test problems. The bilevel approach is found to quickly converge towards the region of efficient parameters. The code for the proposed algorithm can be accessed from the website http://bilevel.org.