Model-assisted approaches for relational reinforcement learning: Some challenges for the SRL community
Tom Croonenborghs, Jan Ramon, Hendrik Blockeel, Maurice Bruynooghe · Lirias · 2006
For a relational reinforcement learning (RRL) agent, learning a model of the world can be very helpful. However, in many situations learning a perfect model is not possible. Therefore, only probabilistic methods capable of taking uncertainty into account can be used to exploit the collected knowledge. It is clear then that RRL offers an interesting testbed for statistical relational learning methods. In this paper, we describe an algorithm taking a middle ground between model-free and model-based (Relational) Reinforcement Learning. A model of the world dynamics in the form of a relational Dynamic Bayesian Network (DBN) is learned incrementally. Empirical results show that sampling the partially learned model outperforms traditional RRL Q-learners. We also focus on a number of open problems. First, it is clear that other SRL techniques, besides the one we are using, could be used just as well. It might be interesting to see what their strengths and weaknesses are in the specific RRL context. In addition, it is typical for our approach that chunks of partial knowledge are obtained, and little is known about how to combine, evaluate and exploit this partial knowledge more efficiently. 1.