Classifier Systems as Linear Probability Models
Gijs Schröder, Johannes C. Textor · Proceedings of the Genetic and Evolutionary Computation Conference · 2025
Classifier systems solve regression and classification problems in high-dimensional spaces by generating and evolving large populations of simple rules; examples include learning classifier systems and artificial immune systems. The properties of these flexible adaptable systems are less well understood than those of more classical machine learning algorithms. Here, we reveal a deep connection between classifier systems and probabilistic models such as Naïve Bayes and Markov chains by showing that all of these can be expressed as generalized linear probability models. This connection shows that any probability distribution can in principle be expressed by a classifier system. We then harness this new perspective to investigate the tradeoff between model complexity and calibration — i.e., the ability to accurately fit the sequence probabilities observed in the training set — for classifier systems applied to sequence probability modeling. Contrasting our results to Markov chains of varying order, we find that a simple model classifier system has a broadly similar complexity-calibration tradeoff. We hope that our approach paves the way for further systematic investigation of the fundamental properties of classifier systems, which could make them more accessible for the machine learning community.