Sigmoids Distinguish More Efficiently Than Heavisides
Eduardo D. Sontag · Neural Computation · 1989
Every dichotomy on a 2k-point set in ℝN can be implemented by a neural net with a single hidden layer containing k sigmoidal neurons. If the neurons were of a hardlimiter (Heaviside) type, 2k − 1 would be in general needed.