Faithful feature extraction by greedy network-growing algorithm
Ryotaro Kamimura, Osamu Uchida · 2004
In this paper, we propose a new computational method for the information theoretic method called greedy network-growing algorithm. The method is called "greedy", because a network with the greedy algorithm tries to absorb as much information as possible from outside. We have so far used the sigmoidal activation function for competitive unit outputs. The method can effectively suppress many competitive units by generating strongly negative connections. However, because methods with the sigmoidal activation function is not so sensitive to input patterns, we have observed that in some cases final representations obtained by the method do not necessarily describe faithfully input patterns. To remedy this shortcoming, we employ the inverse of distance between input patterns and connection weights for competitive unit outputs. As the distance is smaller, competitive units are more strongly activated. Thus, winning units tend to represent input patterns more faithfully than the previous method with the sigmoidal activation function. We applied the new method to animal classification. Experimental results confirmed that more information can be acquired and more explicit features can be extracted by our new method.