Information theoretic rule discovery in neural networks
Ryotaro Kamimura, Taeko Kamimura · 2002
Proposes a new information-theoretic method called structural information, and argues that this new method should be substituted for the traditional competitive method. Structural information control is a more powerful and biologically sounder model, because it uses a soft winner-takes-all model instead of a hard winner-takes-all model. Experiments were conducted to apply the structural information to linguistic rule extraction in which the choice of different donatory verbs must be inferred in an unsupervised way. We found that the structural information control can detect linguistic rules more accurately than the traditional competitive learning method.