Hierarchical State Representation and Action Abstractionsin Q-Learning for Agent-Based Herding
Tao Mao · International Journal of Information and Electronics Engineering · 2012
A primary challenge of agent-based policy learning in complex and uncertain environments is escalating computational complexity with the size of the task space and the number of agents. Nonetheless, there is ample evidence in the natural world that high functioning social mammals learn to solve complex problems with ease. This ability to solve computationally intractable problems stems in part from brain circuits for hierarchical representation of state and action spaces and learned policies arising from these representations. Using such mechanisms for state representation and action abstraction, we constrain state-action choices in reinforcement learning in order to improve learning efficiency and generalization of learned policies within a single-agent herding task. We show that satisficing and generalizable policies emerge, which reduce computational cost, and/or memory resources.