Detection and resolution of learning conflict in backpropagation networks
Derek Edward Kerstetter · 1997
The backpropagation network is one of the most widely used neural classifiers. One impediment to training a backpropagation network is conflicting information in the training patterns. If there is sufficient conflict within the training set, the backpropagation network will enter into a gridlock condition during training. When the network enters into gridlock, the network stops learning. This dissertation presents a method for detecting and overcoming the learning problems caused by conflict in training pattern sets. This method is used to create a new class of neural networks, the auto-partitioning neural networks. Auto-partitioning neural networks are characterized by their ability to detect and rectify learning conflict. Several methods are developed for measuring intra-class conflict for patterns in a given class of the training set. Both the network architecture and training alogrithm are taken into consideration for computing the conflict measures. Based on intra-class conflict measures, patterns of a given class are partitioned into an appropriate number of groups. Two different partitioning methods are developed. Depending on the partitioning method used, the auto-partitioning neural network is called the self partitioning neural network or the recursive partitioning neural network, regardless of the patterns in each group. Because there is little intra-class conflict within a partition, each subnetwork requires few training iterations to reach convergence. After training is completed, the outputs of the subnetworks are combined to produce the final response. This approach is shown to improve classification accuracy, reduce training time and support incremental learning.