A Dynamic Tree Structure for Incremental Reinforcement Learning of Good Behavior
Tomas Landelius, Hans E. Knutsson · 1994
This paper addresses the idea of learning by reinforcement, within the theory of behaviorism. The reason for this choice is its generality and especially that the reinforcement learning paradigm allows sys-tems to be designed, which can improve their behavior beyond that of their teacher. The role of the teacher is to dene the reinforcement function, which acts as a description of the problem the machine is to solve. Gained knowledge is represented by a behavior probability den-sity function which is approximated with a number of normal dis-tributions, stored in the nodes of a binary tree. It is argued that a meaningful partitioning into local models can only be accomplished in a fused space consisting of both stimuli and responses. Given a stimulus, the system searches for responses likely to result in highly reinforced decisions by treating the sum of the two normal distributions on each level in the tree as a distribution describing the system's behavior at that resolution. The resolution of the response, as well as the tree growing and pruning processes, are controlled by a random variable based on the dierence in performance between two consecutive levels in the tree. This results in a system that will never be content but will indenitely continue to search for better solutions. 1