A transformation for implementing localist neural networks
George Rudolph, Tony R. Martinez · 1995
Most Artificial Neural Networks (ANNs) have a fixed topology during learning, and typically suffer from a number of short-comings as a result. Variations of ANNs that use dynamic topologies have shown ability to overcome many of these problems. This paper introduces Location-Independent Transformations (LITs) as a general strategy for parallel implementation of feedforward networks that use dynamic topologies. A LIT creates a set of location-independent nodes, where each node computes its part of the network output independent of other nodes, using local information. This type of transformation allows efficient support for adding and deleting nodes dynamically during learning. This paper deals specifically with LITs for localist ANNs---localist in the sense that ultimately one node is responsible for each output. In particular, this paper presents LITs for two ANNs: a) the single-layer competitive learning network, and b) the counterpropagation network, which combines elements of supervised learning with competitive learning. The complexity of both learning and execution algorithms for both ANNs is linear in the number of inputs and logarithmic in the number of nodes in the original network.