Bayesian online learning in the perceptron.
Ole Winther, Sara A. Solla · 1997
In a Bayesian approach to online learning a simple approximate parametric form for posterior is updated in each online learning step. Usually in online learning only an estimate of the solution is updated. The Bayesian online approach is applied to two simple learning scenarios, learning a perceptron rule with respectively a spherical and a binary weight prior. In the first case we rederive the results for the optimal Hebb-type online algorithm for spherical input distribution.