Intelligent cognition‐based systems approach to multiple‐criteria computerized essay assessment
Shu‐Ling Cheng · Systems Research and Behavioral Science · 2010
Abstract With the advance of computational techniques, computerized assessment becomes increasingly popular due to the advantages of consistency, efficiency and labour‐saving. In computerized assessment, it evaluates the subject's performance based on the experts' assessment criteria and also criteria preferences, thus indicating an effort to implement expert assessment policy. The key element of developing the computerized assessment system is to measure the complexity of an expert's cognitive structures containing assessment criteria and criteria preferences. To achieve this goal, we present an intelligent cognition‐based systems approach which includes three integrated parts to build the computerized assessment model. It first utilizes text‐mining techniques to automatically around important meaning themes that frame as assessment criteria. Human experts' criteria preferences are then derived from the pre‐assessed essays by applying a multiple‐criteria decision analysis. Meanwhile, the optimal parameter set for the assessment system is determined using the genetic algorithm. To demonstrate its effectiveness, the essays of university students majoring in information management are empirically evaluated by the proposed method. The results show that the proposed method can not only effectively model expert cognitive structure but also report high classification accuracy under different assessment settings. Copyright © 2010 John Wiley & Sons, Ltd.