Learning Classes of Real Functions
Martin Anthony, Peter L. Bartlett · Cambridge University Press eBooks · 1999
Introduction This part of the book examines supervised learning problems in which we require a learning system to model the relationship between a pattern and a real-valued quantity. For example, in using a neural network to predict the future price of shares on the stock exchange, or to estimate the probability that a particular patient will experience problems during a surgical procedure, the predictions are represented by the real-valued output of the network. In the pattern classification problems studied in Parts 1 and 2, the ( x, y ) pairs are generated by a probability distribution on the product space X × {0, 1}. In a similar way, we assume in this part of the book that the data is generated by a probability distribution P on X × ℝ. This is a generalization of the pattern classification model, and includes a number of other data-generating processes as special cases. For example, it can model a deterministic relationship between patterns and their labels, where each ( x, y ) pair satisfies y = f(x) for some function f . It can model a deterministic relationship with additive independent observation noise, where y i = f(x i ) + η i , and the η i are independent and identically distributed random variables. It can also model a noisy relationship in which the observation noise variables η i are mutually independent, but the distribution of η i depends on the pattern x i .