The statistical distribution of Boolean gates in two-inputs, one-output multilayered neural networks
Mirta B. Gordon, Pierre Peretto · Journal of Physics A Mathematical and General · 1990
The authors study the probability of implementing Boolean functions in layered neural networks, as a function of the number of hidden units and layers. They show how these probabilities depend on the values allowed to the thresholds, and how they evolve as a function of the number of hidden layers. For two input variables, it is shown that the probability of implementing the EXCLUSIVE OR with one hidden layer remains low, even with a large number of hidden units. In the limit of an infinite number of layers, all the functions become equally probable, but probabilities already reach the asymptotic value within 10% with only five hidden layers.