Incremental state acquisition for Q-learning by adaptive Gaussian soft-max neural network
Hajime Murao, Shinzo Kitamura · 2002
We propose an adaptive Gaussian soft-max neural network to construct a state space suitable for Q-learning to accomplish tasks in continuous sensor space. In the proposed method, a state of Q-learning is defined by a hidden neuron of the neural network which is used to estimate resulting sensor signals of actions. The learning agent starts with single state covering whole sensor space and a new state is generated incrementally by adding a new hidden neuron when difference between the estimated sensor signal and incoming one exceeds a given threshold. Simulation results show that the proposed algorithm is able to construct the sensor space effectively to accomplish the task.