Sparse adaptive memory
Brian K. Flachs · 1995
Pattern recognition is a budding field with many possible approaches. Codebook systems like the k-nearest neighbor classifier, the k-means algorithm, and learning vector quantization all require large numbers of prototype patterns to achieve low error rate recognition. Prototype patterns are an expensive resource for a codebook classifier. This dissertation seeks to reduce the number of prototype patterns required for low error rate recognition by optimizing the positions of the prototypes in pattern space. The optimization is performed by combining backpropagation learning techniques with heuristic algorithms specifically tailored to cooperate with backpropagation learning. A key feature of the sparse adaptive memory learning architecture is the ability to adaptively change the class labeling of the prototypes. This extra degree of freedom allows these algorithms to be developed for on-line operation where training samples are processed one-at-a-time rather than stored and processed in batches. The quiescent properties of the sparse adaptive memory learning algorithms are found to be consistent with the minimal probability of error decision rule. This learning algorithm is compared with several others in the context of a synthetic pattern recognition benchmark, handwritten digit recognition, star classification, and inverted pendulum stabilization. Sparse adaptive memory is shown to require fewer prototype patterns to achieve lower error rates. The number of prototypes required for low error rates is linearly related to the recognition problem's geometric complexity.