On the Statistical Properties of Function Representation with Discrete Variable Basis under Squared Error Loss
Katsuyuki Hagiwara, Naohiro Toda, Shiro Usui, 克幸 萩原, 尚宏 戸田, 支朗 臼井 · Institutional Repositories DataBase (IRDB) · 1995
One of the most important property of 3-layered neural networks is the selectability of the basis functions. In this paper, to focus on the selectability in the context of the regression model, we restricted our attention to function representations in which the basis functions are modified according to the associated discrete parameters. For such function representations, we derived lower and upper bounds for the expectations of the empirical loss and the expected loss with respect to the distribution of the set of samples by taking the squared error as a loss function, provided that the given set samples is a Gaussian noise sequence and the basis functions satisfy the orthonormality condition. Based on these results, we showed that the statistical properties of the function representations with adaptive basis functions is defferent from conventional function representations with fixed basis functions.