Automatic Option Generation in Hierarchical Reinforcement Learning via Immune Clustering
Jing Shen, Guochang Gu, Haibo Liu · 2006
An open problem in hierarchical reinforcement learning is how to automatically generate hierarchies, e.g. options. We consider an immune clustering approach for automatic construction of options in a dynamic environment. The learning agent generates an undirected edge-weighted topological graph of the environment state transitions online. An immune clustering algorithm is then used to partition the state space. A second immune response algorithm is used to update the clusters when a new state being encountered later. Local strategies for reaching the different parts of the space are separately learned and added to the model in a form of options. By our approach, the options not only can be automatically generated but also can be dynamically updated