Towards convergence of learning classifier systems value iteration
Jan Drugowitsch, Alwyn M. Barry · The University of Bath Online Publications Store (The University of Bath) · 2006
In this paper we are extending our previous work on analysing Learning Classifier Systems (LCS) in the reinforcement learning framework [4] to deepen the theoretical analysis of Value Iteration with LCS function approximation.After introducing our formal framework and some mathematical preliminaries we demonstrate convergence of the algorithm for fixed classifier mixing weights, and show that if the weights are not fixed, the choice of the mixing function is significant.Furthermore, we discuss accuracy-based mixing and outline a proof that shows convergence of LCS Value Iteration with an accuracy-based classifier mixing.This work is a significant step towards convergence of accuracy-based LCS that use Q-Learning as the reinforcement learning component.