Online POMDP with Heuristic Search and Sampling Applied to Real Time Strategy Games
Thiago França Naves, Carlos R. Lopes · 2017
Planning and decision making in real-time environments with resource constraints is a complex task. Real time strategy (RTS) games provide a rich domain with all these characteristics, as well as providing a testbed environment. In this work, a Partially Observable Markov Decision Process (POMDP) approach is proposed to deal with the planning and decision making problem in RTS games. The online version of POMDP is modified to work with two decision making fronts, which map macro actions present at different levels of abstraction in the game, the POMDP use heuristics and sample states respectively to manage the number of states and observations, and the approach adapts decisions when the POMDP model changes due to events in the environment. The results show success in planning and decision making in the game, with responses compatible with real-time constraints.