A Survey of Model-Based and Model-Free Methods for Resolving Perceptual Aliasing
Guy Shani · 2004
We focus our attention on agents learning to act in an unknown domain using noisy sensors. Such domains may be modeled by a Partially Observable Markov Decision Process (POMDP) that can be solved optimally. However, when the model of the environment is unknown, most research in the area studies model-free methods — methods that learn to act without learning a model. When the agents ’ sensors provide deterministic output, model-free methods produce close to optimal results. However, as sensor noise increases, the accuracy of such methods decreases. Another, less explored, option is the model-based approach — learning a POMDP model of the world, and computing an optimal solution using the learned model. In this survey we explore model-based and model-free techniques for handling perceptual aliasing. 2