Mutual information maximization by free energy-based competitive learning for self-organizing maps
Ryotaro Kamimura · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · 2008
In this paper, we propose a new information-theoretic approach to self-organizing maps. Information-theoretic methods have been applied to self-organizing maps in several different ways. The methods have been considered to be powerful enough to give simple and practical procedures to replace conventional self-organizing maps. However, computational complexity in terms of mutual information and cooperation has restricted their applications to minimum points. To overcome this situation, we use a free energy similar to that in statistical mechanics in controlling information. By this introduction, computation complexity in mutual information can significantly be reduced. Experimental results showed that feature maps obtained by free energy minimization was significantly similar to those by the conventional SOM. In addition, for some cases, clearer feature maps could be obtained by the free energy method.