Machine Learning | Past and Future
Ray J. Solomonofi · 2009
I will flrst discuss current work in machine learning { in particular, feedforeword Artiflcial Neural Nets (ANN), Boolean Belief Nets (BBN), Support Vector Machines (SVM), Radial Basis Functions (RBF) and Prediction by Partial Matching (PPM). While they work quite well for the types of problems for which they have been designed, they do not use recursion at all and this severely limits their power. Among techniques employing recursion, Recurrent Neural Nets, Context Free Grammar Discovery, Genetic Algorithms, and Genetic Programming have been prominent. I will describe the Universal Distribution, a method of induction that is guaranteed to discover any describable regularities in a body of data, using a relatively small sample of the data. While the incomputability of this distribution has sharply limited its adoption by the machine learning community, I will show that paradoxically, this incomputability imposes no limitation at all on its application to practical prediction. My recent work has centered mainly on two systems for machine learning. flrst might be called The Baby Machine We start out with the machine having little problem speciflc knowledge, but a very good learning algorithm. At flrst we give it very simple problems. It uses its solutions to these problems to devise a probability distribution over function space to help search for solutions to harder problems. We give it harder problems and it updates its probability distribution on their solutions. This continues recursively, solving more and more di‐cult problems. task of writing a suitable training sequence has been made much easier by Moore’s Law, which gives us enough computer speed to enable large conceptual jumps between problems in the sequence. ⁄ Revision of lecture given at AI@50, Dartmouth Artiflcial Intelligence Confer