Functional Approximation of Higher-Order Neural Networks
Jinyan Li, Tommy W. S. Chow · Journal of Intelligent Systems · 1996
The function approximation capabilities of first-order neural networks have been investigated rigorously, but few related works about higher-order neural networks have been reported.In this paper, we proved that higherorder neural networks can approximate any continuous function on a compact set with an arbitrary degree of accuracy, provided that the activation function belongs to C° and is non-polynomial.According to this theory, we know that partially connected higher-order neural networks can approximate any continuous functions as fully connected neural networks can do.Based on this argument, we designed a partially connected higherorder network to compare with the first-order network when maintaining almost the same number of connections.The theory is validated by applying to sunspots series and simulation results show that the partially connected higher-order network exhibits better convergence rate and generalisation ability compared to the first-order network.