On-line Learning under Concept Drift
Pieter Jan Eilers · 2020
Using a modeling framework for the purpose of investigating on-line learning processes in non-stationary environments, we conduct experiments for a number of different situations. We consider the learning of a regression scheme in layered neural networks using sigmoidal and ReLU activation. In all situations, the target, i.e. the regression scheme, changes continuously while the system is trained from a stream of input data. We run Monte Carlo simulations in Student-Teacher scenarios equal number of student and teacher units, K = M. We extend this to the overlearnable case, where K>M. We include weight decay as a from of explicit forgetting and study its effects with regards to drift.