Learning Function Estimation
Sanjeev R. Kulkarni, Gilbert H. Harman · Wiley series in probability and statistics · 2011
This chapter discusses methods for estimation. A best estimate is one that minimizes the expected cost or loss. Nearest neighbor and kernel methods extended in a natural way by suitably averaging the labels yi instead of voting as was done in the case of classification. Multilayer neural networks can be used for estimation in the natural way by using sigmoidal activation rules, although the final output unit can use a simple linear activation rule. Arbitrary continuous functions can be approximated by multilayer networks, and backpropagation can be used to train such networks. The results of probably approximately correct (PAC) learning can be extended to the estimation problem by considering a suitable notion of shattering and pseudo-dimension, which generalize the notion of shattering and VC dimension. Controlled Vocabulary Terms estimator; kernel regression; nearest neighbor classifiers