Paradox-Free Analysis for Comparing the Performance of Optimization Algorithms
Yuan Yan, Qunfeng Liu, Yun Li · IEEE Transactions on Evolutionary Computation · 2022
Numerical comparison serves as a major tool in evaluating the performance of optimization algorithms, especially nondeterministic algorithms, but existing methods may suffer from a ‘cycle ranking’ paradox and/or a ‘survival of the non-fittest’ paradox. This paper searches for paradox-free data analysis methods for numerical comparison. It is discovered that a class of sufficient conditions exist for designing paradox-free analysis. Rigorous modeling and deduction are applied to a class of profile methods employing a filter. It is thus further discovered and proven that algorithm-independent filter conditions can prevent cycle ranking and survival of non-fittest paradoxes from occurring. By adopting an algorithm-independent filter, popular profile methods such as the ‘modified data profile method’‘, the accuracy profile method’, and ‘the operational characteristics zones method’ can be paradox free in comparing or benchmarking the performance of optimization algorithms.