Combining PCA and entropy criterion to build ANN's architecture
Al-jun Li, Siwei Luo, Yunhui Liu, Zhi-Hong Nan · 2004
Designing of artificial neural network (ANN or NN)'s architecture is a fundamental problem, which draws researchers' concern. This paper proposes PCA and entropy as a criterion to select neuron and provides a method, PCA-ENN, to build NN. First, according to the similarity or equivalence between decision tree (DT) and NN, PCA-ENN adopts PCA to extract new feature attributes. Second, PCA-ENN selects the best cut point for each new attribute by entropy criterion and selects the best attribute for classification as a neural unit. Then specifies the connection weights between input units and outer inputs by coefficients obtained from PCA and specifies the biases of input units as the best cut points. At the same time, PCA-ENN constructs the hidden and output layer units, and initializes the connection weights of units. PCA-ENN cannot only build architecture of NN effectively, but also make NN's incremental learning possible.