Study of methods for model reduction in transition systems
Sudeshna Sarkar, A.V. Subramaniam, R. Neogi · 2002
We consider the reinforcement learning problem where the agent interacts with the environment by taking actions. The agent receives some reward from the environment and changes state as a result. The problem is to find a good policy in large transition systems such that the agent's value function is maximized. If the transition system is Markovian, there exist several algorithms for finding the optimum policy in such systems. However, finding the optimum policy may take considerable time for large systems, and the learning algorithm will not converge if we allow a smaller number of iterations. Our objective is to investigate methods of reducing the state space of large transition systems, and to evaluate the effect of model reduction on getting a good policy in reasonable time. We conjecture that when we have limited time, it may be wise to reduce the transition system and then apply our learning algorithm to the smaller system.