Patterns bit by bit. An Entropy Model for Rule Induction
Silvia Radulescu, Frank N. K. Wijnen, Sergey Avrutin · 2019
From limited evidence, children track the regularities of their language impressively fast and they infer generalized rules that apply to novel instances. This study investigated what triggers the inductive leap from memorizing specific items and statistical regularities to extracting abstract rules. We propose an innovative entropy model that offers one consistent information-theoretic account for both learning the regularities in the input and generalizing to new input. The model predicts that rule induction is an encoding mechanism triggered by the discrepancy between input complexity (entropy) and the encoding power of the human brain (channel capacity). In two artificial grammar experiments with adults we probed the effect of input complexity on rule induction. Results showed that as the input entropy increases, the tendency to infer abstract rules increases gradually.