Two Classes of Abstract Modes about Markov Decision Processes
Zhizheng Zhang · 2008
Using Markov decision processes on abstract level,one can compactly and declaratively represent complex Markov decision processes.And one can solve real-world problems that generally have very large state space with regular MDPs.This paper introduces central concepts of two classes of abstract MDPs:structured mode and generalized mode.Then precise or approximate algorithms for looking for optimal policy for abstract MDPs are reviewed,including an algorithm that is totally different from regular Markov decision processes:A relational upgrade of the Bellman update operation.Finally,the paper gives conclusions of history work and suggests future work.In this way,people are able to get an intensive,comprehensive and in-depth understanding of abstract MDPs.