Validating information systems requirements with prototypes and scenarios
J. David Naumann, David L. Bahn · University of Minnesota eBooks · 1999
Determining the functional requirements for an information system is a significant problem for information systems design because it is dependent upon effective communication between the system designer and the prospective user. Incomplete requirements determination for a new information system leads to increased modification costs or even project failure. Before constructing an information system, the designer usually summarizes functional requirements into representations that the user reviews and validates. Requirements representations can range from textual descriptions to operational prototypes that demonstrate key functions. Requirements review may be conducted at various stages of the development process to obtain user feedback and critique. A final validation review commonly precedes approval to construct the new system. This dissertation used a pilot study followed by a laboratory experiment to investigate two methods for enhancing user feedback during requirements validation: prototyping and scenarios. Prototyping is a method that facilitates learning about an intended information system through user review and critique of working models. Scenarios are story-like descriptions of potential functions for an intended system. Scenarios are a method to direct the user's review of requirements and stimulate feedback. A completely randomized experimental design examined the effects of both prototypes and scenarios upon user feedback during requirements validation. User review of interactive prototypes was contrasted with the review of paper-based mock-ups of a prototype. Employing scenarios to direct user review of requirements was contrasted with a loosely guided requirements review. 104 graduate and undergraduate students in three sections were organized into 52 pairs of subjects who enacted the roles of a user and a systems designer engaged in requirements validation for an intended online grocery shopping system. Contrary to hypothesized expectations, the results indicated that scenarios stymied rather than stimulated some kinds of user feedback. Furthermore, prototypes and paper-based mock-ups were found to be roughly equivalent in their stimulation of feedback. This study is among the first to experimentally substantiate that scenarios affect user feedback in requirements communication. The results also suggest that systems designers should present a combination of prototypes and abstract models to users when discussing requirements. Directions for further experiments on requirements representations are indicated.