ON CONVERGENCES OF ASYMPTOTICALLY OPTIMAL DISCRIMINANT FUNCTIONS FOR PATTERN CLASSIFICATION PROBLEMS
Masafumi Watanabe · Bulletin of Mathematical Statistics · 1974
This is a continuation of the paper [11] and is concerned with the pattern classification problem related to "learning with a teacher". In [11], in the case when the optimal discriminant function is assumed to belong to the $ L^2 $ space and the case when that is assumed to be uniformly continuous, we gave algorithms, which were applications of the stochastic approximation method, for constructing the asymptotically optimal estimates, and investigated the convergence (mean convergence and almost sure convergence) of the algorithms. But we did not consider the rate of almost sure convergence. In this paper we shall discuss the convergence of the algorithm in the case when the "optimal discriminant function" (o. d. f.) is continuous and the rate of the almost sure convergence. This paper consists of five sections. In Section 2, we shall give definition of the o. d. f. and of asymptotically optimal estimates to the o. d. f., and we shall prepare several lemmas to be used throughout subsequent sections. In Section 3, we shall treat the case when the o. d. f. is continuous, and give an algorithm which is more general than the form in [10]. And we shall discuss the almost sure convergence and the mean convergence of asymptotically optimal estimates. In Section 4, we shall give some inequalities concerning the rates of convergences.