A new view of the formal entropy as a measure of interdependence and its application to pattern recognition
Hiroshi Watanabe · 2002
A novel entropic measure of interdependence among groups of objects is introduced, and applied to some clustering problems successfully. Most practical algorithms to carry out clustering tasks are based upon the notion of distance between two objects. For the purpose of taking 'more-than-two-elements correlation' into account, S. Watanabe (1936; 1969) introduced an entropic measure of similarity and cohesion among groups of objects, and proposed a method of interdependence analysis. His method has a number of theoretical merits, but sometimes it does not work. The article clarifies the nature of its difficulty, proposes a possible modification of the method, and discusses the advantages of the novel approach by showing examples.