Approximate realization of identity mappings by three‐layer neural networks
Ken-ichi Funahashi · Electronics and Communications in Japan (Part III Fundamental Electronic Science) · 1990
Abstract Recently, applications of neural networks to information compression, e.g., image data compression, have been studied. This paper discusses these problems from a viewpoint of the approximate realization of identity mappings by the use of three‐layer neural networks. The relationship between statistical properties of data, hidden units number and the infimum of approximation error also is clarified theoretically by the study of the relation with principal component analysis in the case in which the hidden layer generally has nonlinear units. As a result, it is proved that the capability of information compression by three‐layer networks does not exceed the capability of the K‐L transformation method. Moreover, the phenomenon of generalization in this case is explained theoretically.