Usability engineering and evaluation models of practice: Who does what, when, where and why? A dynamic group discussion
Brian Hilligoss, Bill Thomas, Michelle Rago, Chris Testa, Jeri Ann Payne, Carol Stennett, Teal Anderson, Andrew Dillion · Proceedings of the American Society for Information Science and Technology · 2005
In acknowledgement of the importance of the principles of user-centered design, more and more organizations are looking to incorporate usability engineering and testing in their design and evaluation processes, often utilizing internal resources. While many agree on the importance of usability, there are a wide variety of models of practice. In some cases, a single individual is tasked with handling the entire job. In other cases, many professionals are involved, each with specific roles defined. Some approaches are very informal, while others are very structured and established within the fabric of the organizational culture. Is one approach better than another? Is it dependent on the organization, the digital product itself, or some other factor? Who should be involved and at what point? To what extent is methodological rigor necessary or generalizability possible? How can practitioners and researchers in usability inform each others work? This panel will discuss how usability engineering and testing are conducted in the design and evaluation of digital libraries at several different organizations, both public and private. The benefits and drawbacks of various models will be discussed as we investigate a variety of scenarios and differing needs. In order to highlight the similarities and differences between the models of practice presented by the panelists, an interactive format will be used. The session will begin with a representative from each organization briefly describing the organization and its digital products. Once this background information has been shared, the moderator will ask a number of usability-related questions, which panelists from each of the organizations will answer from their own experiences and within the contexts of their organizations. Throughout the session audience members will be encouraged to pose questions as well and join in this dynamic group discussion. This format will enable us to explore how any given aspect of usability may be viewed and approached differently by different organizations faced with their own unique confluence of constraints and concerns. In addition, an experienced usability researcher will weigh in to provide a research perspective. The flexible, dynamic nature of this session will allow the content to be shaped, in part, by the interests of the audience. However, a number of questions reflective of concerns common among usability practitioners will be addressed. The following are a few such questions and may serve to stimulate initial discussions. Who is responsible for usability? How and where does usability fit into the design process? What data gathering and analysis methods should be used? What constitutes a usability “problem”? How should usability test results be reported? What are the barriers to developing a user-centered design culture and how do we overcome these barriers? In short, how do we “practice” usability? The panelists represent a variety of skills, roles and experiences in practice, as well as, research, while their organizations range from government institutions to academic research libraries to for-profit corporations. This mix will enable the examination of multiple models of practice and a preliminary analysis of the various factors that shape these models for better or worse. The Library of Congress recently redesigned the user interface for its American Memory site (http://memory.loc.gov/ammem/), which houses over 9 million of the Library's digital items -including photographs, sound recordings, videos, maps, and manuscripts - all free to the public. This significant undertaking involved the contributions of numerous professionals with a wide variety of skills, including usability engineering, information architecture and project management. The Library of Congress team will discuss methods used, the importance of the integration of usability and information architecture, the role of a project manager in usability evaluation efforts, and other vital issues pertaining to the measuring and improvement of usability for one of the nation's premier digital library collections. Sirsi Corporation develops products for the ever-changing needs of libraries and consortia of all types and sizes. Recently, staff have been engaged in the creation a web-based user interface that will enable library patrons a simpler, more intuitive and more efficient means of accessing library information and services from their desktops. To help with this effort, they have formed a usability team, comprised of a cross section of talent, including backgrounds in librarianship, Web development, information architecture, usability and human interaction, marketing communications and product management. Presenters will discuss the process of forming the team and its efforts, both formal and informal, to measure usability. As a Web usability specialist working in an academic library, this panelist has conducted numerous usability evaluations, utilizing many different data collection methods sometimes working alone, sometimes with a team, and sometimes with partners outside of the University. Among the many projects evaluated are the Sheridan Libraries Web Site, the Johns Hopkins University Libraries Catalog, and online collections of medieval manuscripts, journal articles, sheet music, and art images. The appropriate application of usability methods to digital library projects, the role of the usability specialist in an academic library setting, and the training of usability team members will be discussed. Usability practices have evolved over the last two decades but often lack the rigor and formalism that researchers demand. As a result, adherence to usability principles and, more broadly, user-centered design, is easy to claim but often difficult to demonstrate convincingly to research audiences. Andrew Dillon has written and consulted extensively on usability methods. He will contribute to the discussion of the strengths and weaknesses of the various approaches outlined by the other panelists and share insights from theoretical and methodological analyses of evaluation practices in an attempt to bridge the research and practice gap within usability engineering.