Backpropagation representation theorem using power series
M.-S. Chen, MICHAEL T. MANRY · 1990
A representation theorem is developed for backpropagation neural networks. First, it is assumed that the function to be approximated,F(x) for the vectorx, is continuous and has finite support, so that it can be approximated arbitrarily well by a multidimensional power series. The activation function, sigmoid or otherwise, is then approximated by a power-series function of the net. Basic building-block subnetworks, realizing the monomial or product of the inputs, are implemented with any desired degree of accuracy. Each term in the power series forF(x) is realizable using a building block, and each building block has one hidden layer. Hence, the overall network has one hidden layer