Simple Principles of Metalearning
Jürgen Schmidhuber, Jieyu Zhao, Marco Wiering · 1996
The goal of metalearning is to generate useful shifts of inductive bias by adapting the current learning strategy in a "useful" way. Our learner leads a single life during which actions are continually executed according to the system's internal state and current policy (a modifiable, probabilistic algorithm mapping environmental inputs and internal states to outputs and new internal states). An action is considered a learning algorithm if it can modify the policy. Effects of learning processes on later learning processes are measured using reward/time ratios. Occasional backtracking enforces success histories of still valid policy modifications corresponding to histories of lifelong reward accelerations. The principle allows for plugging in a wide variety of learning algorithms. In particular, it allows for embedding the learner's policy modification strategy within the policy itself (self-reference). To demonstrate the principle's feasibility in cases where conventional reinforcemen...