Efficient Learning Algorithms for Neural Networks (ELEANNE): Extensions, Revisions and Improvements
Nicolaos B. Karayiannis, Gopathy Purushothaman · IETE Journal of Research · 1996
This paper presents revisions, improvements, and extensions of the family of ELEANNE algorithms, proposed recently for training multilayered neural networks. The improved ELEANNE algorithms employ a special procedure which compensates the recursive Hessian matrix inversion for its arbitrary initialization. The ELEANNE algorithms, which were originally developed by minimizing the quadratic error criterion, are extended in this paper to train feed-forward neural networks by maximizing the relative entropy criterion. The comparison of the proposed algorithms with their original version indicated that the revisions of the algorithms proposed in this paper improve significantly their convergence, reduce the occurrence of local minima during the training process, and make the algorithms less sensitive to parameters provided by the user. Experiments based on the double spiral problem, a well-known nontrivial training task, also indicated that the improved ELEANNE algorithms perform considerably better than the corresponding gradient-descent-based algorithms in terms of both convergence rate and training speed.