Generalised mixtures of experts, independent expert training, and learning classifier systems
Jan Drugowitsch, Alwyn M. Barry · The University of Bath Online Publications Store (The University of Bath) · 2007
We present a generalisation to the Mixtures of Experts model that introduces prior localisation of the experts as part of the model structure, and as such relates them strongly to the evolutionary computation ML method known as Learning Classifier Systems.While the introduced generalisation allows specification of more complex localisation patterns, identifying good models becomes more difficult.We approach this tradeoff by introducing a new training schema that makes fitting a single model computationally less demanding and shifts the importance to searching the space of possible model structures, guided by approximate variational Bayesian inference to fit the model and find the model evidence.We demonstrate model search for simple non-linear curve fitting tasks by sampling from the model posterior, as a proof-of-concept alternative to the genetic algorithm used in Learning Classifier Systems for that purpose.