Multilayer Neural Networks: One Or Two Hidden Layers?
Graham Brightwell, Claire Kenyon, Hélène Paugam‐Moisy · 1996
We study the number of hidden layers required by a multilayer neural network with threshold units to compute a function f from R d to f0; 1g. In dimension d = 2, Gibson characterized the functions computable with just one hidden layer, under the assumption that there is no "multiple intersection point" and that f is only defined on a compact set. We consider the restriction of f to the neighborhood of a multiple intersection point or of infinity, and give necessary and sufficient conditions for it to be locally computable with one hidden layer. We show that adding these conditions to Gibson 's assumptions is not sufficient to ensure global computability with one hidden layer, by exhibiting a new non-local configuration, the "critical cycle", which implies that f is not computable with one hidden layer. 1 INTRODUCTION The number of hidden layers is a crucial parameter for the architecture of multilayer neural networks. Early research, in the 60's, addressed the problem of exactly rea...