Probabilistic Regression Using Basis Function Models ; CU-CS-975-04
Gregory Z. Grudić · CU Scholar (University of Colorado Boulder) · 2004
Our goal is to accurately estimate the error in any prediction of a regression model.We propose a probabilistic regression framework for basis function regression models, which includes widely used kernel methods such as support vector machines and nonlinear ridge regression.The framework outputs a point specific estimate of the probability that the true regression surface lies between two user specified values, denoted by y 1 and y 2 .More formally, given any y 2 > y 1 , we estimate the Pr(y 1 ≤ y ≤ y 2 |x, f (x)), where y is a true regression surface, x is the input, and f (x) is the basis function model.Thus the framework encompasses the less general standard error bar approach used in regression.We assume that the training data is independent and identically distributed (iid) from a stationary distribution, and make no specific distribution assumptions (e.g.no Gaussian or other specific distributions are assumed).Theory is presented showing that as the number of training points increases, estimates of Pr(y 1 ≤ y ≤ y 2 |x, f (x)) approach the true value.Experimental evidence demonstrates that our framework gives reliable probability estimates, without sacrificing mean squared error regression accuracy.