Training speed-up methods for neural networks applied to word recognition
G. Thierer, A. Krause, Heidi Hackbarth · 2002
Two methods for speeding up the training of neural networks for word recognition are presented. The idea of the first method is to reduce the number of training patterns. The number of vocabulary repetitions can be cut down from seven to one if a pretrained network is used as the basis for further learning instead of a randomly initialized network. The second method does not need a pretrained network. Instead, training is performed alternatively with the entire training set and a subset thereof. This saves unnecessary backpropagation cycles for patterns that have already been learned. Depending on the data material, network training time is reduced at least by a factor of seven in the first case, and by a factor of two in the second case. Moreover, the error rate for a 100-word vocabulary can be lowered by one fourth by applying the second method.>