Quantitative Performance Prediction for Rule-Based Expert Systems*
Valerie Barr · 1998
Often a rule-based system is tested by checking its performance on a number of test cases with known solutions, modifying the system until it gives the correct results for all or a sufficiently high proportion of the test cases. However, the performance on the test cases may not accurately predict performance of the system in actual use. We present a method for making a more accurate performance prediction based on the performance on test cases, coverage of the rule-base by the test cases, how representative the test data is of the population on which the rule-base will be used, and the likelihood of occurrence of different kinds of test cases in the larger population. Introduction Over the last 20 years, during which time there has been considerable development and use of knowledgebased systems for medical decision support, there has been a heavy emphasis on functional analysis, addressing, among others, the question: does the system give the results we expect on the test cases? Gene...