Neural networks for abstraction and reasoning: Towards broad generalization in machines
Mikel Bober-Irizar, Soumya Banerjee · 2024
For half a century, artificial intelligence research has attempted to reproduce the human qualities ofabstraction and reasoning - creating computer systems that can learn new concepts from a minimal setof examples, in settings where humans find this easy. While specific neural networks are able to solve animpressive range of problems, broad generalisation to situations outside their training data has provedelusive. In this work, we look at several novel approaches for solving the Abstraction & Reasoning Corpus (ARC).This is a dataset of abstract visual reasoning tasks introduced to test algorithms on broad generalization.Despite three international competitions with $100,000 in prizes, the best algorithms still fail to solve amajority of ARC tasks. The best solvers today rely on complex hand-crafted rules, without using machinelearning at all. We revisit whether recent advances in neural networks allow progress on this task, orwhether an entirely different class of models are required. First, we adapt the DreamCoder Neurosymbolic reasoning solver to ARC. DreamCoder automaticallywrites programs in a bespoke domain-specific language to perform reasoning, using a neural networkto mimic human intuition. We present the Perceptual Abstraction and Reasoning Language(PeARL) language, which allow DreamCoder to solve ARC tasks, and propose a new recognition modelthat allows us to significantly improve on the previous best implementation. We also propose a new encoding and augmentation scheme that allows large language models (LLMs) to solve ARC tasks, and find that the largest models can solve some ARC tasks. LLMs are able to solvea different group of problems to state-of-the-art solvers, and provide an interesting way to complementother approaches. We perform an ensemble analysis, combining models to achieve better results than any system alone. Finally, we publish the arckit Python library to make future research on ARC easier.