Reinforcement learning for Golog programs with first-order state-abstraction
D. Beck, Gerhard Lakemeyer · Logic Journal of IGPL · 2012
A special feature of programs in the action language Golog are non-deterministic constructs such as non-deterministic choice of actions or arguments. It has been shown that in the presence of stochastic actions and rewards reinforcement learning techniques can be applied to obtain optimal choices for those choice-points. In order to avoid an explosion of the state space, an abstraction mechanism is employed that computes first-order state descriptions for the given program. Intuitively, the idea is to generate abstract descriptions that group together states for which the expected reward of executing the program is the same. A current limitation is that a non-deterministic choice of arguments can be handled only if the possible candidates are known in advance. In this article we show how this restriction can be lifted. We also show how a first-order variant of binary decision diagrams can be used to efficiently compute first-order state abstractions. Moreover, we give a completely declarative specification of a learning Golog interpreter that incorporates the presented state-abstraction mechanisms.