Neural Networks, Approximation Theory, and Dynamical Systems(Structure and Bifurcation of Dynamical Systems)
Ken-ichi Funahashi, Yuichi Nakamura · Institutional Repositories DataBase (IRDB) · 1992
Abstr act In this paper, firstly we discuss the capability problem of feedforward neural networks from the aspect of approximation theory.Secondly we prove that any finite time trajectory of a given n-dimensional dynamical system can be approximately realized by the internal state of output units of continuous time recurrent neural networks with $n$ output units, some hidden units, and an appropriate initial conditions.The essential idea of the proof is to embed the n-dimensional dynamical system into a higher dimensional one by the approximate realization theorem of continuous mappings of three-layer neural networks.As a corollary, we also show that any continuous curve can be approximated by outputs of a recurrent neural network. \S \S 1. $\ln$ trod ucti onNeural networks are divided into two types namely, feedforward networks and recurrent networks from the architectural aspect.For the former networks without feedback connection, ever since the back propagation learning algorithm was proposed by Rumelhart-Hinton-Williams [19], a lot of application was made mainly to the static information processings such as pattern recognition.On the theoretical capability of this networks, Funahashi [9], and Cybenko [8] proved mathematically that a given continuous mapping on a compact set can be realized by three-layer feedforward neural networks with any precision.In this paper we discuss the related problems from the aspect of approximation theory.The nonlinear dynamical behavior of the latter networks is suitable for the spatio- temporal information processings.The theoretical studies for the recurrent networks 数理解析研究所講究録 第 804 巻 1992 年 18-37