Analog versus Discrete Neural Networks
Bhaskar DasGupta, Georg Schnitger · Neural Computation · 1996
We show that neural networks with three-times continuously differentiable activation functions are capable of computing a certain family of n-bit boolean functions with two gates, whereas networks composed of binary threshold functions require at least omega(log n) gates. Thus, for a large class of activation functions, analog neural networks can be more powerful than discrete neural networks, even when computing Boolean functions.