Online state space generation by a growing self-organizing map and differential learning for reinforcement learning
Akira Notsu, Koji Yasuda, Seiki Ubukata, Katsuhiro Honda · Applied Soft Computing · 2020
In this research, we develop a method integrating a growing self-organizing map and differential learning system for online reinforcement learning which adaptively builds the state structure. In the conventional method, models and information on the environment are required beforehand, whereas the proposed method automatically estimates the state transitions from differentials of input signals and from these builds the state space without reference to prior information on the environment. Also, since it is an online learning method, the proposed method requires less computation and no batch memory. Through numerical experiments, we show that the proposed method has the same performance as the conventional method with the information given and that the learning time is shortened by abstraction of the state space.