XCS with dynamic sized experience replay for memory constrained applications
Anmol Prakash Surhonne, Manuel Wensauer, Florian Maurer, Thomas J. Wild, Andreas Herkersdorf · Proceedings of the Genetic and Evolutionary Computation Conference Companion · 2024
The eXtended Classifier System (XCS) is the most widely studied classifier system in the community. It is a class of interpretable AI which has shown strong capability to master various classification and regression tasks. It has also shown strong performance in certain multi-step environments in the reinforcement learning domain. XCS consists of a population of classifiers of size N which is decided at design time. The population size N is typically large to provide room for the learning and generalization mechanism of XCS. Experience replay (ER) is a popular technique in reinforcement learning which significantly improves the learning of the agents. ER uses a replay memory of fixed size which is defined at design time. Typically XCS has been trained on high-performance computers or servers which have near to no limitations on memory. XCS when applied to embedded applications or IoT devices are constrained by the memory consumption. This memory constraint affects the population size and the size of the replay memory in ER. In this work, we propose XCS with dynamic sized experience replay, where the size of the replay memory is resized inversely proportional to the number of macro-classifiers in the population to maximize the performance within a memory constraint.