An empirical evaluation of the cognitive interview for elicitation of episodic knowledge during expert systems development
Janette W. Moody · 1993
The development of expert systems depends upon capturing the expert's knowledge, either in the form of rules (for rule-based expert systems) or cases (for case-based expert systems). In either instance, the elicitation of knowledge from an expert has been consistently characterized as the primary stumbling block or bottleneck in the expert systems development process. Various techniques have been suggested for knowledge elicitation and attempts have been made to match the elicitation technique to the type of knowledge sought. These methods have been criticized for their lack of theoretical foundations regarding how humans process and store knowledge, and few have been empirically tested. Despite the lack of empirical support for a particular technique, the literature cites the interview as the most commonly used method of knowledge elicitation. Unfortunately, the interview is notable for eliciting knowledge that is imprecise, incomplete, and inconsistent because the knowledge sought is often subconscious and not readily available by the expert for articulation. These issues of deficiencies in memory retrieval have been faced in other disciplines. Accordingly, this study is based on the premise that a theoretically grounded technique utilized successfully in other disciplines will prove useful in knowledge elicitation during expert systems development. The technique is the Cognitive Interview which is structured around five principles of memory retrieval: context reinstatement, focused retrieval, extensive retrieval, varied retrieval, and multiple representations. The research project consisted of a field experiment utilizing a randomized two-group design. Subjects were reference librarians in academic institutions who were interviewed by Expert Systems students trained in both the Cognitive Interview and the standard knowledge elicitation interview. The two dependent variables measured were efficiency and effectiveness. Both variables were found to be significantly higher under the Cognitive Interview, leading the researcher to conclude that knowledge engineers can benefit from including the Cognitive Interview during the knowledge acquisition process.