Toward second-order generalisation
R.S. Neville, P.C.K. Luk · 2002
Generalisation in artificial neural networks may be cast into two basic categories, 'standard' and 'higher-order'. We view 'standard' generalisation as a means to interpolate and extrapolate data. A two-layer perceptron network performs 'standard' generalisation when if has learnt a function from a set of discretised vectors that represent a given function. The trained network can then interpolate and extrapolate between the data points it was initially trained on. We define 'higher-order' generalisation as generally of a more abstract nature. For example if one trains a unit to learn a function, then one manipulates the weight matrix of the unit. If the transformed weight matrix allows the unit to perform the inverse function then this is a 'higher-order' generalisation. The article relates how one can perform a set of transforms on the nets weight matrix to enable the transformed net to perform a type of 'higher-order' generalisation.