Recurrent Networks as Models of Short Term Memory
Matt Jones, Thad A. Polk · 2002
The class of recurrent networks known as attractor networks is known to exhibit behaviors relevant to modeling human memory processes – notably content-addressable memory, storage of repeated inputs as stable patterns (under Hebbian learning), and maintenance of information (as activity) over time. In addition, these networks provide a natural account of the effect of similarity on interference in recall. However when looked at in finer detail there are some ways in which traditional attractor networks fail as models of human short-term memory. In particular, information in human short-term memory decays over time unless it is rehearsed, rather than remaining indefinitely. Also, under Hebbian learning traditional attractor networks have particular trouble learning correlated patterns. Here we investigate some variations on the classic framework which make it more appropriate for modeling human STM. We show (1) how adjusting the threshold of continuously-valued units can lead to networks which maintain activity information temporarily, but decay over time, (2) how noise in learning and/or input leads the similarity structure of the set of stored patterns to be reflected in the distribution of recall errors, and (3) how adding a timedelayed anti-correlative component to the learning rule provides robustness against highly correlated patterns and varying levels of input. These ideas have been incorporated in a model of serial recall that explains many aspects of human behavior on that task, and have also been used in newer simulations that learn temporal properties of the environment.