Limit cycles in neural networks for information processing
James A. Kottas, Cardinal Warde · Massachusetts Institute of Technology eBooks · 1991
This thesis considers the use of limit cycles in neural networks as a means of storing and processing information. Methods and applications for both continuous-time and discrete-time cycles are discussed. Both types can represent symbols of information or logical states of a system. Discrete-time cycles also can represent temporal relationships between symbols when the individual symbols are represented by fixed points. As an example application, a simple finite state machine based on continuous-time limit cycles (LC-FSM) is described and demonstrated via computer simulations. The states, inputs and outputs of the machine are cycles. The features required of a LC-FSM are (1) a medium that is capable of storing and accessing many limit cycles and (2) a method for recognizing and implementing transitions from one cycle to another. These features are obtained through the spectral back-propagation (SBP) training algorithm and the self-oscillating neural network (SONN) model. The SBP algorithm is a modified form of the conventional recurrent back-propagation algorithm. It can train a neural network to learn pairs of input-output sequences. It computes the Fourier series of the output sequences and compares that with the Fourier series of the desired output sequences to generate a spectral error criterion. In addition to the weights, this approach allows the interconnects to have trainable time delays. This feature is very useful for training a network to learn to recognize a transition condition. The SBP algorithm also permits the cells in the network to have finite bandwidth as approximated by a first-order low-pass filter. Before being applied to the LC-FSM, the SBP algorithm is derived and characterized with complete generality. The SONN model is useful as an associative memory for the LC-FSM. The SONN exhibits many unique limit cycles for both constant and oscillatory inputs without requiring any training. It is a recurrent hierarchical structure of identical feedforward networks called levels. Each level has an off-center, on-surround canonical interconnect topology. Using computer simulations, the SONN is analyzed and characterized, also with complete generality. The results show the SONN is very tolerant of static variations in its network parameters ($>$ $\pm$20%). Furthermore, variations in the interconnect time delays produce more diverse cycle shapes. An optical architecture for implementating a SONN using spatial light modulators also is presented. Finally, a working LC-FSM consisting of 3 states and 8 transitions is constructed using a particular SONN for both the memory and to generate the limit cycle input to the LC-FSM. The SBP algorithm was used to train 8 transition-detecting networks to recognize each transition condition. These networks could respond within 3 cycle periods and had a relative phase sensitivity of 7$\sp\circ.$ The simulated LC-FSM showed that an input cycle and current-state cycle can induce different transitions depending upon their relative phase. (Copies available exclusively from MIT Libraries, Rm. 14-0551, Cambridge, MA 02139-4307. Ph. 617-253-5668; Fax 617-253-1690.)