Feature Discovery by Information Loss
Ryotaro Kamimura · Journal of Computers · 2009
Abstract — In this paper, we propose a new approach called information loss to feature detection in competitive learning. The information loss is defined by the difference between a full network and a network without some elements. If this deletion significantly decreases the amount of information contained in a network, the elements are considered to be important and are expected to play a very important role. The method was applied to artificial and symmetric data to show the features extracted by the information loss. Then, we applied the method to the classification of OECD countries. The experimental results confirmed that the method was efficient enough to detect main features comparable to those detected by the conventional SOM. Index Terms—mutual information, information loss, feature detection, competitive learning, self-organizing maps information loss now becomes more general and more flexible than the previous one. In Section 2, after explaining information-theoretic competitive learning, we present how to compute the information loss. In Section 3, we present experimental results on two problems. In the first problem, we use artificial data to show intuitively the features extracted by the information loss. In the second example, the classification of OECD countries, we try to show that experimental results obtained by the information loss are comparable or superior in some cases to those obtained by the conventional computational methods, such as the U-matrix. I.