Testing children's information retrieval systems: Challenges in a new era
Jamshid Beheshti, Dania Bilal, Allison Druin, Andrew Large · Proceedings of the American Society for Information Science and Technology · 2010
The purpose of this Panel is to discuss and elaborate on testing and evaluation challenges facing the designers of new and novel information systems for children.Since 2002, ASIST has hosted 16 papers, panels, and posters on information retrieval systems, digital libraries, and user interfaces for children * Traditionally, much of the research on IRS has been experimental, where researchers strive to control as many variables as possible in a carefully regulated environment to , but very little discussion has been devoted to methodological challenges.Many new systems are designed for today's children, the "Digital Natives" (those born after 1989), who may process information differently from their predecessors.Digital natives often are well versed in using computer games, such as those designed for the X-Box 360 and the Nintendo Wii.However, a panel of experts, meeting recently to discuss the information behavior and needs of the new generation of users, concluded that a research agenda is urgently required to investigate the "characteristics and preferences of this tech savvy group that surprisingly lacks basic skills in information evaluation and retrieval" (Radford et al, 2008, p. 2).Developing new and novel information retrieval mechanism for these digital natives, however, require new ways of testing and evaluation of the system. BACKGROUNDResearch has shown that the overwhelming majority of information retrieval systems (IRS) such as Google are unsuitable for children.Many of the problems typically encountered by children in information retrieval includes determining appropriate concepts, translating these concepts to keywords that the information retrieval system can understand, finding appropriate synonyms, and distinguishing homonyms (Bilal, 2000;Druin et al, 2009;Large, Nesset, Beheshti, 2008).