A study of the negative transfer problem in artificial neural networks
Adel M. Abunawass · 1990
In this study a second sequential learning problem in artificial neural networks is reported. The problem is known as the negative transfer problem. The negative transfer problem is the degradation in the learning ability and performance of artificial neural networks over successive training sessions. Three experiments were designed and conducted to study the effect of negative transfer on the back-propagation model. The effect of negative transfer on the error rate of the network and on the number of sweeps needed for learning was studied. The error rate and the number of sweeps increase as a function of successive training sessions. The two main causes of the negative transfer were found to be: the unbounded growth of the weight values; and the expansion of the search space for subsequent training sessions. Four methods found in the literature were developed to treat the negative transfer problem. The methods were unsuccessful in solving the problem. A new sequential learning framework is introduced. The framework is known as the adaptive memory consolidation. The framework consists of two independent processes. The first process is a weight clipping process which insures that the weights do not attain extreme values. This process is applied to each weight of the network at every sweep. The second process is a memory consolidation process which stabilizes and constrains the search space. This process is applied to the weights between training sessions. The adaptive memory consolidation framework uses past learning to improve future learning. This makes the framework completely adaptive and guided by the network. Two methods were developed based on the adaptive memory consolidation framework. The methods are: the adaptive decay method; and the standard weights method. The methods are computationally simple and inexpensive to use. The methods were successful in eliminating and minimizing the effect of the negative transfer problem. Additionally the methods improved the performance of the network when used in conjunction with shaping schedules.