On-line learning from a finite training set: A solvable model
B López, Manfred Opper · Europhysics Letters (EPL) · 2000
We discuss the problem of on-line learning from a finite training set with feedforward neural networks. Defining a modified learning rule, which randomly chooses inputs and weights to be updated, the dynamics of learning can be treated within a diffusion approximation in the thermodynamic limit. No assumption on the generation of data is made. Explicit results for the stationary distribution and relaxation times can be found for a network with linear transfer function. Assuming self-averaging of the diffusion term, a general relation between on-line learning and batch learning with an effective temperature is established.