Reexpressing problematic optimization data
Mark Wineberg, Sebastian Lenartowicz · Proceedings of the Genetic and Evolutionary Computation Conference · 2017
In current evolutionary computation (EC) praxis, the two most common metrics of success are time to completion and, when completion is not possible, best achieved fitness. As neither commonly produces normally-distributed data, non-parametric statistical analysis techniques are usually employed when performing comparisons and one-way analysis-of-variance (ANOVA). This approach, however, does not incorporate the analysis of interactions when considering multiple factors as is done using a multi-way ANOVA. Unfortunately, stable non-parametric multi-way ANOVA does not yet exist. Furthermore, even non-parametric one-way ANOVA techniques are counter-indicated if the variance around each treatment is not homogeneous, as is often the case for EC data. Instead, statisticians typically employ data reexpression to normalize the dataset, and then apply traditional parametric techniques. In this paper, we introduce the RePOD workflow, which utilizes existing and novel statistical techniques in order to perform more powerful analyses on EC data. By applying this workflow, we find that Box-Cox reexpression is applicable to a variety of EC systems, problems, parameter settings, and performance metrics. We also note that, when time to completion and best achieved fitness can be defined on the same dataset, subtle differences appear in analysis, which becomes especially apparent when examining factor interactions.