Assessing the contributions of multiple experts in a structured knowledge acquisition process
Roberta M. Roth, Gary A Wicklund · 1990
Knowledge acquisition is widely recognized as a bottleneck in the development of expert systems, particularly when using multiple experts. This research proposed a structured expert system development model separating knowledge acquisition from programming activities. Within this framework, a structured knowledge acquisition process suitable for use by single experts or by multiple experts in group settings was proposed. This process was designed to support initial conceptualization of the knowledge structure. This study's objectives were: assessing the value of the structured knowledge acquisition process as a knowledge acquisition approach, and evaluating the value of using multiple experts as compared to an individual expert as knowledge sources. The structured knowledge acquisition process consisted of a Delphi study and a workshop consisting of the nominal group technique, interpretive structural modeling, and ideawriting. This structured knowledge acquisition process was implemented in an actual expert system development project, involving an expert group and an individual expert in a parallel process. Data was analyzed to determine the contributions of the individual expert compared to that of the group, and to assess the effectiveness of the approach as a knowledge acquisition process. The Delphi study was successful in surfacing and identifying relationships among key domain concepts. The Delphi results enabled the problem scope to be refined prior to the workshop session. Through the three Delphi rounds, the expert group and individual expert demonstrated a common core of shared concepts, with the group contributing a larger number of unique ideas. The workshop session permitted domain concepts to be generated and structured. Interpretive structural modeling was especially beneficial in delineating a complex relation structure among the concepts. While the group again contributed more unique ideas, a substantial core of shared knowledge was also identified. This study demonstrated the implementation and value of a structured knowledge acquisition process. This process was useful in developing a preliminary understanding of the domain knowledge prior to system construction. Multiple experts contribute more complete coverage of domain concepts. The areas common knowledge provide a foundation from which to build the knowledge base.