Catastrophic Forgetting in Connectionist Networks: Causes, Consequences and Solutions
Robert M. French · 1999
All natural cognitive systems, and, in particular, our own, gradually forget previously learned information. Consequently, plausible models of human cognition should exhibit similar patterns of gradual forgetting old information as new information is acquired. Only rarely (see Box 3) does new learning in natural cognitive systems completely disrupt or erase previously learned information. In other words, natural cognitive systems do not, in general, forget catastrophically. Unfortunately, however, this is precisely what occurs under certain circumstances in distributed connectionist networks. It turns out that the very features that give these networks their much-touted abilities to generalize, to function in the presence of degraded input, etc., are the root cause of catastrophic forgetting. The challenge is how to keep the advantages of distributed connectionist networks while avoiding the problem of catastrophic forgetting. In this article, we examine the causes, conseque...