A learning as the evolution of representation
Pasquale Caianiello · 1990
We formulate the learning problem as the construction of a semantics for the reality under observation. We provide rigorous definitions of the terms involved which allow the identification of a novel concept of structure: the alphabet. We propose that a learning mechanism is one which constructs such structures as representations of the given data. The structural description constructed are based on the two concepts of code and classification used as data compression mechanisms. We provide means for evaluating the efficiency of the encodings created using ideas from information theory. We reduced learning to an optimization problem and we suggest that the mechanisms proposed work at any descriptive level. We show how a slight shift of focus allows learning to be conceived as a natural evolution of a physical system to its ground states. The applications discussed include an architecture and a learning rule for neural nets as well as a procedure for grammatical inference.