The Sparchunk code: a method to build higher-level structures in a sparsely encoded SDM
Gunnar Sjödin · 2002
An important property for any memory system is the ability to form higher-level concepts from lower-level ones in a robust way. This process is in the article called chunking. It is also important that such higher-level concepts can be analyzed, i.e., broken down into their constituent parts. This is called probing and clean-up. These issues have previously been treated for vectors of real numbers and for dense binary patterns. Using sparse codes instead of dense ones has many advantages. The paper shows how to define robust chunking operations for such sparse codes. It is shown that a sparse distributed memory (SDM) model using sparse codes and a suitable activation mechanism can be used as a clean-up memory. It is proved that the retrieval of the constituent parts can be made arbitrarily exact with a growing memory. This is so even if we let the load increase to infinity.