Modeling action feasibility in POMDPs with boolean-valued preconditions
Caroline Ponzoni Carvalho Chanel, Florent Teichteil-Königsbuch, Guillaume Infantes, Patrick Fabiani · 2011
In automated planning, action preconditions are boolean-valued formulas, which check whether a given action is feasible in a given state. While crucial for realistic applications where dangerous actions in some states must be discarded, preconditions have never been formally considered in POMDPs. One reason is that preconditions are defined over states whereas decisions depend on the current belief of the agent. Simply defining preconditions over beliefs is not sufficient because, as each belief is possibly defined over many states, there is no guarantee to prevent the agent from applying an infeasible damaging action. Augmenting the observation space with feasible actions does not help more, since the optimization process still maximizes the value of the current belief over all existing actions in the model. Thus, we propose an extension of the traditional POMDP model that, by means of an additional information step semantically different from standard observations, allows the agent to know the set of feasible actions before deciding the best action to apply. Without requiring a full knowledge of the current state, this extended model leads to a significant modification of the decision process, for which we provide a proved optimization scheme. We also compare the value and the execution paths of policies optimized either with the standard model or with our extended one, and show that our policies are always safe and gather more rewards at execution.