Generalization of Rules by Neural Nets
Giorgio Parisi, František Slanina · Europhysics Letters (EPL) · 1992
We investigate the generalization abilities of various types of rules for both feed-forward and symmetric Hopfield neural network. The networks of the size of 24 neurons are taught to perform the rule on a small set of examples using simulated annealing optimization. Results show a rather small dependence of the type of network, but a significant difference in generalizing different rules is found. This suggests the possibility of defining the complexity of a rule to a certain extent independently of physical implementation. A simple power-law behaviour of generalization for a small number of examples n e is found to be G ∼ n e 1/2 for AND rule, while G ∼ n e for adding.