The implementation of Q-learning for problems in continuous state and action space using SOM-based fuzzy systems
Mu‐Chun Su, Lu-Yu Chen, De-Yuan Huang · Annual Conference on Computers · 2006
In reinforcement learning, there is no supervisor to critically judge the chosen action at each step. The learning is through a trial-and-error procedure interacting with a dynamic environment. Q-learning is one popular approach to reinforcement learning. It is widely applied to problems with discrete states and actions and usually implemented by a look-up table where each item corresponds to a combination of a state and an action. However, the look-up table implementation of Q-learning fails in problems with continuous state and action space because an exhaustive enumeration of all state-action pairs is impossible. In this paper, an implementation of Q-learning for solving problems with continuous state and action space using SOM-based fuzzy systems is proposed. Simulations of training a robot to complete two different tasks are used to demonstrate the effectiveness of the proposed approach. Reinforcement learning usually is a slow process. In order to accelerate the learning procedure, a hybrid approach which integrates the advantages of the ideas of hierarchical learning and the progressive learning to decompose a complex task into simple elementary tasks is proposed.