Consistent population control
Vasil Khalidov, Maxime Oquab, Jérémy Rapin, Olivier Teytaud · 2019
Resampling methods, based on averaging the fitness of several clones, are the classical solution for dealing with noise. Population control has been proposed as a different tool for faster convergence of evolution strategies when the variance does not vanish around the optimum. However, we show that convergence may not hold even in the case of centered noise and construct a counterexample with variance dissymmetry, i.e. more variance on one side of the optimum than on the other. We propose a fix termed consistent population control and formally derive uniform constraints on deviations between averages and expectations within the proposed algorithm under either subgaussianity or finite variance assumptions on measurement noise. We prove convergence guarantees of consistent population control, verify it experimentally and show the effectiveness of population control in direct policy search, either with our fix or, in overparameterized cases, without our fix.