Computing confidence measures and marking unreliable predictions by estimating input data densities with MLPs
Lars Kindermann, A. Lewandowski, Michael Tagscherer, Peter Protzel · 2003
In this paper we present a method to compute the distribution of input vectors with a standard MLP architecture. By training the network on all data vectors with a target output of "1" and an additional set of random vectors with a zero target, the network is able to approximate the local density of seen input data for any point in the input space. While densities can be computed with high precision by a number of specialized algorithms, this fast and very easy implementable method allows easily to evaluate the outputs of a network used for function approximation or classification. If networks are queried with data outside the training set, the result usually will be unpredictable. But determining if the current point lies within the reliable area is a classification problem comparable to the main problem itself. By using three parallel networks of the same type and structure it is possible to evaluate the precision and validity of predictions as well with minimal additional effort: in addition to the network used for prediction we use one to predict the absolute error and another to determine the input density as an alert signal.