A General Structural Learning of Connectionist Models Using Forgetting
Masumi Ishikawa · IEEJ Transactions on Electronics Information and Systems · 1992
The backpropagation through time, a learning algorithm for recurrent networks, has a drawback of being unable to provide structural information, such as locations of feedback loops in a network. I have previously proposed a structural learning algorithm with forgetting of link weights for hierarchical networks. By eliminating unnecessary links due to forgetting, the structural learning can generate a skeletal structure reflecting inherent regularity in training patterns.Proposed here is a general structural learning algorithm with forgetting for recurrent networks by combining the above two algorithms. This algorithm can generate a skeletal structure for recurrent networks based on input and output sequences with no prior structural information.Four kinds of Jordan networks are selected as typical examples of recurrent networks including feedback and self loops. The proposed algorithm can, in most cases, rediscover the original Jordan network structure solely from input and output sequences. This well demonstrates the effectiveness of the proposed algorithm in rediscovering the original network structure for recurrent networks.