Constrained information maximization
Ryotaro Kamimura, Shinobu Nakanishi · 2002
We propose a constrained information maximization method for the explicit interpretation of networks' behaviors. In the constrained information maximization, the information, defined as the decrease of the uncertainty of hidden units, is maximized with a fixed total unit activity. By the constrained information maximization, we can easily control a network configuration to obtain appropriate final states of the maximum information, in which all the components such as units or connections of a network, are explicitly determined for the interpretation. Thus, the interpretation of the network behavior is much easier. We applied the constrained information maximization method to the acquisition of rules for past tense forms of an artificial language. The constrained information maximization enabled us to interpret the network behavior explicitly, keeping the generalization performance improved.