Induction and Learning of Finite-State controllers from Simulation (Extended Abstract)
Matteo Leonetti, Luca Iocchi, Subramanian Ramamoorthy · 2012
We propose a method to generate agent controllers, repre-sented as state machines, to act in partially observable envi-ronments. Such controllers are used to constrain the search space, applying techniques from Hierarchical Reinforcement Learning. We define a multi-step process, in which a sim-ulator is employed to generate possible traces of execution. Those traces are then utilized to induce a non-deterministic state machine, that represents all reasonable behaviors, given the approximate models and planners used in simulation. The state machine will have multiple possible choices in some of its states. Those states are choice points, and we defer the learning of those choices to the deployment of the agent in the actual environment. The controller obtained can therefore adapt to the actual environment, limiting the search space in a sensible way.