CMAC neural network with improved generalization property for system modeling
Gábor Horváth, Tamas J. Szabo · 2003
This paper deals with some important questions of the CMAC neural networks. CMAC - which belongs to the family of feed-forward networks and is considered as an alternative to MLPs - has some attractive features. The most important ones are its extremely fast learning capability and the special architecture that lets effective digital hardware implementation possible. Although the CMAC architecture was proposed in the middle of the seventies quite a lot open questions have been left even for today. Among them the most important ones are its modeling and generalization capabilities. While some essential questions of its modeling capability were addressed in the literature no detailed analysis of its generalization properties can be found. Neural networks with good generalization capability play important role in system modeling. This paper shows that CMAC may have significant generalization error, even in one-dimensional case, where the network can learn exactly any training data set. The paper shows that this generalization error is caused mainly by the architecture and the training rule of the network. It presents a mathematical analysis of the generalization error, derives a general expression of this error and proposes a modified training algorithm that helps to reduce this error significantly.