Estimating Missing Data and Determining the Confidence of the Estimate Data
Jaisheel Mistry, Fulufhelo Vincent Nelwamondo, Tshlidzi Marwala · 2008
A Computational Intelligence approach to estimate missing data makes use of Autoassociative Neural Networks (ANN) and a stochastic optimization technique. The ANN captures interrelationships within data and the optimization technique estimates probable values that are used as inputs to the ANN. The optimum estimate is one that has a minimum influence on the output of the ANN. A method to determine the confidence of this estimate is presented in this paper. An ensemble of ANNs with a Multi Layer Perceptron architecture is collected using Bayesian training methods. The percentage of the most dominant estimate values is used as a confidence measure. The South African antenatal seroprevalence survey data is used and the HIV status of the patients is estimated. It was found that the missing data could be estimated with an overall accuracy of 68% and the confidence ranges between 50% and 97%. Estimates that have a confidence exceeding 70% have 88% estimation accuracy.