Hierarchical heuristic forward search in Stochastic domains
Nicolas Meuleau, Ronen I. Brafman · 2007
Many MDPs exhibit an hierarchical structure where the agent needs to perform various subtasks that are coupled only by a small sub-set of variables containing, notably, shared resources. Previous work has shown how this hierarchical structure can be exploited by solving several sub-MDPs representing the different subtasks in different calling contexts, and a root MDP responsible for sequencing and synchronizing the subtasks, instead of a huge MDP representing the whole problem. Another important idea used by efficient algorithms for solvingflatMDPs,suchas(L)AO*and(L)RTDP,is to exploit reachability information and an admissible heuristics in order to accelerate the search by pruning states that cannot be reached from a given