A model of recurrent neural network with high capacity
Chang-Jiu Chen · 1993
A recurrent neural network is a one-layer network with lateral-connection. It feedbacks the output vector to the input during processing and stops when the output vector is stable. A recurrent neural network can act as an associative memory, and the capacity is an important criterion in this kind of memory. Therefore, the study of the capacity of recurrent neural network attracted many researchers. In our study of the capacity of recurrent neural networks, we find an upper bound of the network, which is a function of r, the minimum attraction radius of all attraction regions. Many works focused on the topic of the capacity of recurrent neural network. However, there is a big gap between the capacity of those works and the upper bound. In this dissertation, we propose a High Capacity (HC) model which explores the capacity, $${n\choose \lfloor{n\over2}\rfloor},$$of recurrent neural network. It is the high capacity. The HC model reduces the benefit of noise tolerance to obtain a high capacity. We analyze the properties resulted from the proposed model. The comparisons of the proposed model with other works are discussed in terms of the number of capacity and the behavior of the network. We also discuss several methods regarding the intersection and subtraction of two HC models. Hopfield neural network is one of the most widely studied model among the recurrent neural networks. We propose an efficient simulation model for Hopfield neural networks based on the condition that the number of memorized vectors, m, is smaller than the number of neurons, n, in the network.