Modeling How and When Learning Happens in a Simple Fault-Finding Task
Frank E. Ritter, Peter A. Bibby · 2001
We have developed a process model that learns in multiple ways using the Soar chunking mechanism while finding faults in a simple control panel device. The model accounts very well for measures such as problem solving strategy, the relative difficulty of faults, average fault-finding time, and, because the model learns as well, the speed up due to learning when examined across subjects, faults, and even series of trials for individuals. However, subjects tended to take longer than predicted to find a fault the second time they completed a task. To examine this effect, we compared the model's sequential predictions -- the order and relative speed that it examined interface objects -- with a subject's performance. We found that (a) the model's operators and subject's actions were applied in basically the same order; (b) during the initial learning phase there was greater variation in the time taken to apply operators than the model predicted; (c) the subject appeared to spend time checking their work after completing the task (which the model did not). The failure to match times on the second time seeing a fault may be accounted for by the subject spent checking their work whilst they learn to solve the fault-finding problems. The sequential analysis reminds us that though aggregate measures can be well matched by a model, the underlying processes that generate these predictions can differ.