Dynamical analysis of LVQ type learning rules
Anarta Ghosh, Michael L. Biehl, Barbara Hammer · University of Groningen research database (University of Groningen / Centre for Information Technology) · 2005
Abstract- Learning vector quantization (LVQ) constitutes a powerful and simple method for adaptive nearest prototype classification which has been introduced based on heuristics. Recently, a mathematical foundation by means of a cost function has been proposed which, as a limit case, yields a learning rule very similar to classical LVQ2.1 and also motivates a modification thereof which shows better stability. However, the exact dynamics as well as the generalization ability of the LVQ algorithms have not been investigated so far in gen-eral. Using concepts from statistical physics and the theory of on-line learning, we present a rigorous mathematical investigation of the dynamics of LVQ type classifiers in a prototypical scenario. Interestingly, one can observe significant differences of the algorithmic stability and generalization ability and quite unexpected behavior for these only slightly different variants of LVQ. Key words- Online learning, LVQ, thermodynamic limit, order parameters.