Decomposition Methods for Solving Finite‐Horizon Large MDPs
Bouchra El Akraoui, Cherki Daoui, Abdelhadi Larach, Khalid Rahhali · Journal of Mathematics · 2022
Conventional algorithms for solving Markov decision processes (MDPs) become intractable for a large finite state and action spaces. Several studies have been devoted to this issue, but most of them only treat infinite‐horizon MDPs. This paper is one of the first works to deal with non‐stationary finite‐horizon MDPs by proposing a new decomposition approach, which consists in partitioning the problem into smaller restricted finite‐horizon MDPs, each restricted MDP is solved independently, in a specific order, using the proposed hierarchical backward induction (HBI) algorithm based on the backward induction (BI) algorithm. Next, the sub‐local solutions are combined to obtain a global solution. An example of racetrack problems shows the performance of the proposal decomposition technique.