Finite-size effects and optimal test set size in linear perceptrons
David Barber, David Saad, Peter Sollich · Journal of Physics A Mathematical and General · 1995
Fluctuations in the test error are important in the learning theory of finite-dimensional systems as they represent how well the test error matches the average test error. By explicitly finding the variance of the test error due to randomness present in both the data set and algorithm for a linear perceptron of dimension n, we are able to address such questions as the optimal test set size. Where exact results were not tractable, a good approximation is given to the variance. We find that the optimal test set size possesses a phase transition between linear and 2 / 3 power-law scaling in the system size n.