Dynamical neural network for nonlinear modeling, prediction, and signal processing
Nagib Hakim · 1992
This thesis presents and studies a class of dynamical discrete-time recurrent neural networks for nonlinear modeling and filtering. The class includes as special cases several models previously applied to signal processing and time series prediction. Batch and recursive learning algorithms are derived to allow the identification of the optimal model parameters. The batch method is based on simulated annealing and gradient descent, while the recursive algorithm uses a Gauss-Newton prediction error method. Optimal architecture selection is also addressed and incorporation of the Akaike Information Criterion in the error function is discussed. Computer simulations illustrate the learning and generalization performance of the algorithm on a variety of problems in identification, prediction, and signal processing. These include prediction of deterministic time series generated by the logistic and Mackey-Glass equations, and a stochastic nongaussian uncorrelated time series. Other simulations include FSK demodulation and modeling of an autonomous system, the Van der Pol oscillator. In order to further characterize the proposed nonlinear class of models, the Volterra theory of nonlinear systems is extended to neural networks and new formulae are presented to compute the Volterra expansions of both feedforward and recurrent architectures. The results are used to define a class of nonlinear systems accurately represented by a specific neural network model. The Volterra expansion is also shown to be useful in assessing learning, and architecture selection of the neural network. The dynamical neural network algorithm is then applied to cursive script online character recognition. A set of recurrent neural networks is trained to segment and recognize a string of handwritten script characters. A decision algorithm compares the different network outputs and reconstructs the string of letters. Performance of this system is assessed and compared to other current recognizers. The results presented in this thesis demonstrate the suitability of discrete-time recurrent neural networks to perform nonlinear modeling and signal processing tasks. This work could serve as a basis for more complex studies, both theoretical and applied.