Performance Measures for Adaptive Decisioning Systems
Robert Y. Levine, Timothy S. Khuon · Defense Technical Information Center (DTIC) · 1991
Performance measures are derived for data-adaptive hypothesis testing by systems trained on stochastic data. The measures consist of the average performance of the systems over an ensemble of training sets. The uncertainties derivable from training sets represents an irreducible uncertainty inherent in the learning procedure. Data-adaptive system estimates are contrasted with classical hypothesis testing, in which optimum tests are based on an assumed data model. In addition, a performance estimate for the maximum a posteriori probability (MAP) N-hypothesis test is derived based on a neural-net formulation of the test. The performance of adaptive systems on a binary test of uniformly distributed data is compared with the data-adaptive and MAP estimates. The adaptive systems considered are linear extrapolation from data (LINEXT) and a back-propagation neural net (BPNN).