Projective visualization: learning to simulate from experience
Marc Goodman · 1995
There are many domains in which the ability to predict future states of the world faster than real-time is desirable. Many of these domains are characterized by their dynamic and continuous nature. Experience in such domains is typically low-level and often noisy. One approach to this problem is to frame the problem as a learning task, i.e., how can an agent in an environment learn to predict the effects of her actions through observation. While such predictions may not be completely accurate since they are based on incomplete experience, it is possible that they may be good enough to allow the agent to perform effectively. This is the approach taken by projective visualization, a technique for learning to project the effects of one's actions into the future based on prior experience. Projective visualization uses a large set of inductively generated decision trees, one for each feature of the case representation, to project a case into the future. The projected case can then serve as a basis for further projection, in a technique called projective simulation. This work describes PVC scLUS, the algorithm used to build case projectors, evaluates the effectiveness of projective visualization, and discusses the application of projective visualization to controlling the actions of an autonomous agent and to simulation in an industrial process-control setting. Further, it is proposed that the architecture for projective visualization is the basis for a cognitive model of human imagery with ties to the mental manipulation of objects, mental practice, perception, and navigation. It is shown that the error rate for projective simulation is linear for small projection windows, and forms a kneed-over curve for longer projection windows. Error rate is also shown to be inversely proportional to the size of the training set. Finally, it is demonstrated that projective visualization can be used to improve the performance of an autonomous agent.