Scaling relationships in back-propagation learning: dependence on training set size
Gerald J. Tesauro · Complex Systems · 1987
We st ud y th e amount of ti me needed to learn a fixed t rain ing se t in the back-pro pagation proced ure for learning in multi-layer ne ural network models. The task chosen was 32-bit parity, a hi gh order fu nct ion for wh ich memor iza ti on o f specific in p u t- out put pairs is necessary. For small t raining sets, the learning time is consistent with a ~-power law depen dence on the nu mber of patterns in the t ra ining set. For lar ger training set s, t he learn ing t ime dive rges at a critical t ra ining set size which appears to be related to the st orage capacity of t he network.