Scale Equalized Higher-order Neural Networks
Chien-Ming Lin, Keng-Hsuan Wu, Jung-Hua Wang · 2006
This paper presents a novel network, called scale equalized higher order neural network (SEHNN) based on concept of scale equalization (SE). We show that SE is particularly useful in alleviating the scale divergence problem that plagues higher order networks. SE comprises two main processes: setting the initial weight vector and conducting the matrix transformation. An illustrative embodiment of SEHNN is built on the Sigma-Pi network (SPN) applied to task of function approximation. Empirical results verify that SEHNN outperforms other higher order networks in terms of computation efficiency. Compared to SPN, and Pi-Sigma network (PSN), SEHNN requires less number of epochs to complete the training process.