Measurement criteria for neural network pruning
S.S. Erdogan, Geok See Ng, Pak Ki Chan · 2002
A new measure based on hidden-output node activation is proposed for measuring the relevance of hidden nodes in a neural network. The concept has been successfully applied for pruning in several classification problems. The experiments indicate that redundant nodes are pruned down resulting in optimal network topologies. The measure has been compared to the one proposed by Kamimura-Nakanishi (see IEICE Trans. Inf. & Syst., vol.E78-D, no.4, p.484-9, 1995) and also used in the context of a modified cost function where an additional penalty function is added to steer the direction of the hidden node's activation in the process of learning.