Incremental learning with sleep - learning of noiseless datasets
Koichiro Yamauchi, Norihiro Kobayashi · 2004
Presents a model-based incremental learning system for noiseless datasets that realizes the following two abilities: 1) learning a new instance perfectly when the system encounters the instance by chance, without forgetting old memories; 2) Model-selection for reduction of redundant hidden units. The system basically has two types of radial basis function networks: a fast-learning network (F-Net) and a slow-learning network (S-Net). The system memorizes new instances quickly by using the F-Net during wake phase, in a manner like that of k-nearest neighbors (k-NN), while reducing redundant hidden units by using the S-Net during sleep phase. The system alternately repeats these two phases. Like humans, the system does not learn new instances during sleep. Several benchmark tests show that the new system learns instances quickly, as does k-NN, but uses only about 10% to 50% of the resources of k-NN.