A Novel Fast Sarsa Algorithm Based on Value Function Transfer
FU Qi-min · Dianzi xuebao · 2014
Knowledge Transfer has gradually became a research hot pot in machine learning,which tries to transfer the knowledge from the historical tasks to the target task in order to speed up the convergence rate and improve the performance of algorithms. With respect to the slow convergence rate of traditional reinforcement learning algorithms,this paper proposed to transfer the value function between different similar learning tasks with the same state space and action space,which tries to reduce the needed samples in the target task and speed up the convergence rate. Based on the framework of on-policy Sarsa algorithm,combined with the value function transfer method,this paper put forward a novel fast Sarsa algorithm based on the value function transfer—VFT-Sarsa. At the beginning,the algorithm uses Bisimulation metric to measure the distance between states in target task and historical task on the condition that these tasks have the same state space and action space,transfers the value function if the distance meets some condition,and finally executes the learning algorithm. At the end,apply the proposed algorithm in Random Walk,compared with Sarsa algorithm,Q-Learning and QV algorithm,the results show that the proposed algorithm can get a better convergence rate with a good performance.