Using multiplicative algorithms to build cascade correlation networks
Nigel Duffy · 2003
Cascade correlation has been shown to learn effectively, producing small networks and low generalization error. However, there remain difficulties with this approach. Cascade correlation can produce networks with large depth and large fan-in. We propose the use of a multiplicative learning algorithm to address these problems. Experimental results indicate that these algorithms may produce sparse weight vectors. Furthermore, theoretical results indicate that these algorithms behave substantially differently from the usual additive algorithms such as gradient descent and Quickprop. It is hoped that by combining these two approaches an effective neural network algorithm will result. We attempt to validate this and motivate further research.