Neural models of supervised and self-supervised learning
Gail A. Carpenter, Gregory P. Amis · 2009
This thesis explores and develops three families of neural systems, each based on Adaptive Resonance Theory (ART), that model how humans learn pattern classes from teachers, how they later adapt their knowledge from experience, and how these models may be formulated for applications. The first project defines a new self-supervised learning paradigm and a new neural network called self-supervised ARTMAP. Self-supervised learning integrates knowledge from a teacher (labeled patterns), knowledge from the environment (unlabeled patterns), and knowledge from internal model activation (self-labeled patterns). Self-supervised ARTMAP incorporates new knowledge from unlabeled data without destroying correct knowledge previously acquired from labeled data. By basing predictions on known features and focusing learning on new features and confident predictions, self-supervised ARTMAP improves testing accuracy on illustrative low-dimensional problems and on more realistic high-dimensional benchmarks. The second project updates an existing neural network, default ARTMAP, to adhere more closely to basic design principles and to improve performance in applications. Default ARTMAP employs winner-take-all activation during training and distributed activation during testing. Search mechanisms of winner-take-all ARTMAP systems are designed to guarantee that each input will make a correct prediction if represented immediately after its training presentation. That is, the network learns from its mistakes, passing the so-called Next Input Test. While distributing activation during testing generally improves accuracy, an input for which a winner-take-all prediction is correct could make an erroneous prediction using distributed activation. The new default ARTMAP 2 model adds a distributed Next Input Test during training, resulting in increased test-set accuracy without significantly decreased code compression. The third project uses distributed ARTMAP to model thirty related human experiments on pattern learning and classification. In these experiments, during the testing phase, subjects display a characteristic pattern of errors on items that they had learned to classify perfectly during training. Many cognitive models have explored this phenomenon, but these systems do not model how an individual learns or forgets through time. Distributed ARTMAP provides such a model, and also clarifies both psychological and neurobiological data.