Inverse PCA method for weight initialization in CMLP network
Mikko I. Lehtokangas · 2003
We have developed a method for finding the minimum or near minimum number of centroid units for the centroid based multilayer perceptron (CMLP) network. When near minimum number of units is used the problem of weight initialization becomes a more dominant factor in the training process. Even a single poorly initialized weight can cause one of the units to become useless in the network operation. However, in a near minimum sized network all the units must function properly. The purpose of this study is to address the problem of weight initialization in a CMLP network where minimum number of centroid units is used. Based on the inverse principal component analysis, we propose an efficient initialization method for the centroid units. In addition, we describe an initialization method for the MLP part of the CMLP. The benchmark simulations demonstrate that the proposed initialization scheme can significantly improve the convergence. The cost of initialization is relatively small compared to the actual training process.