The Invisible Han: How Evaluation Guides AI Research
Paul R. Cohen, Adele E. Howe · 1988
EVALUATION SHOULD BE A MECHANISM OF PROGRESS BOTH WITHIN AND ACROSS AI RESEARCH PROJECTS. FOR THE INDIVIDUAL, EVALUATION CAN TELL US HOW AND WHY OUR METHODS AND PROGRAMS WORK, AND SO TELL US HOW OUR RESEARCH SHOULD PROCEED. FOR THE COMMUNITY, EVALUATION EXPEDITES UNDERSTANDING OF AVAILABLE METHODS AND SO THEIR INTEGRATION INTO FURTHER RESEARCH. IN THIS PAPER, WE PRESENT A FIVE STAGE MODEL OF AI RESEARCH AND DESCRIBE GUIDELINES FOR EVALUATION THAT ARE APPROPRIATE FOR EACH STAGE. THESE GUIDELINES, IN THE FORM OF EVALUTION CRITERIA AND TECHNIQUES, SUGGEST HOW TO PERFORM EVALUATION. WE CONCLUDE WITH A SET OF RECOMMENDATIONS THAT SUGGEST HOW TO ENCOURAGE EVALUATION OF AI RESEARCH.