Smoothing regularizers for projective basis function networks
John E. Moody · 2018
Smoothing regularizers for radial basis functions have been studied extensively, but no general smoothing regularizers for projective basis functions (PBFs), such as the widely-used sigmoidal PBFs, have heretofore th been proposed. We derive new classes of algebraically-simple m-order smoothing regularizers for networks N T of projective basis functions f(W, :r) = 5: big [,c v 5 + v/0] + u0, with general transfer functions g[.]. These regularizers are: RG(m,m) = y}u}ll,Jll 2m- GlobalForm RL(m,m) = y}u}ll,Jll 2m LocalForm With appropriate constant factors, these regularizers bound the corresponding mt*-order smoothing integral Of(W,a: ) 2 In the above expressions, {v j} are the projection vectors, W denotes all the network weights {u j, u0, v j, v0}, and (x) is a weighting function (not necessarily the input density) on the D-dimensional input space. The global and local cases are distinguished by different choices of (x).