Pruning with generalization based weight saliencies: λOBD, λOBS
Lars Kai Hansen, Jan Otto Larsen · Neural Information Processing Systems · 1995
The purpose of most architecture optimization schemes is to improve generalization. In this presentation we suggest to estimate the weight saliency as the associated change in generalization error if the weight is pruned. We detail the implementation of both an O(N)-storage scheme extending OBD, as well as an O(N2) scheme extending OBS. We illustrate the viability of the approach on prediction of a chaotic time series.