On the Uselessness of Finite Benchmarks to Assess Evolutionaryand Swarm Methods
Pablo Rabanal, Ismael Rodrı́guez, Fernando Rubio · 2015
We argue against the usage of known finite benchmarks to compare the performance of swarm and evolutionary methods. The key of our criticism is that these methods support a huge set of parameter values and available sub-steps which can be selected and used, and this huge set of choices provides enough versatility to enable an ad-hoc tuning of the method to the particular inputs to be solved (which does not imply properly solving any input not considered in the benchmark). As an alternative, we propose using random input generators rather than known finite benchmarks.