Approximate Policy Iteration for Generalized Semi-Markov Decision Processes: an Improved Algorithm
Emmanuel Rachelson, Patrick Fabiani, Frédérick Garçia · 2008
Abstract. In the context of time-dependent problems of planning under uncertainty, most of the problem’s complexity comes from the concurrent interaction of simultaneous processes. Generalized Semi-Markov Decision Processes represent an efficient formalism to capture both concurrency of events and actions and uncertainty. We introduce GSMDP with ob-servable time and hybrid state space and present an new algorithm based on Approximate Policy Iteration to generate efficient policies. This al-gorithm relies on simulation-based exploration and makes use of SVM regression. We experimentally illustrate the strengths and weaknesses of this algorithm and propose an improved version based on the weaknesses highlighted by the experiments. 1