Acceleration of back propagation through initial weight pre-training with delta rule
G. Li, Hussein Alnuweiri, Yulin Wu, Hongbin Li · 2002
A training strategy for backpropagation (BP) neural networks, named delta pre-training (DPT), is proposed. The core of the training strategy is based on pre-training the initial weights for BP networks using the delta rule, instead of using random values. After pre-training, the normal BP training procedure is carried out to complete network training. With the DPT, the convergence rate for training BP networks can be significantly improved. Since the DPT deals only with initial weight settings, most variations of the standard BP algorithm can be combined with the DPT so as to further speed up convergence. With regards to on-chip learning in VLSI implementations, only a little additional circuitry is required for the pre-training phase with the DPT. Simulation results using the proposed training method show its superiority over previous methods.>