A linear time natural evolution strategy for non-separable functions
Yi Sun, Tom Schaul, Faustino J. Gomez, Jürgen Schmidhuber · 2013
We present a novel Natural Evolution Strategy (NES) variant, the Rank-One NES (R1-NES), which uses a low-rank approximation of the search distribution covariance matrix. The algorithm allows computation of the natural gradient with cost linear in the dimensionality of the parameter space, and excels in solving high-dimensional non-separable problems.