Web Search Engines Evolve to Meet Challenges
Karen R. Díaz, Nancy O’Hanlon · Reference & User Services Quarterly · 1999
Finding information on the Web can challenge the most experienced searchers. Librarians teach patrons that choosing the correct Web tool is half of the battle. We recommend use of selective directory-type resources such as Yahoo! or LookSmart for general queries or to get acquainted with resources available on a topic, and large, crawler-built indexes like AltaVista or HotBot for more specific questions. But as professional researchers, we know that there are still a number of other key choices to be made along the road to an answer. For example, knowing which Web index to begin with and when to move on to a meta-search site such as Webcrawler, which will run the against multiple sources simultaneously, are skills acquired from often frustrating trial-and-error experiences at the reference desk. Fortunately for ail of us, these popular tools are evolving, based on extensive research into user needs and behaviors. Providers are finding unique ways to improve results for the large Web indexes. In this column, we offer a summary of some important trends in this area. Smarter Tools Developers of Web indexes (or search engines) are working to build more intelligence into responses. The most promising new tool in this regard is Jeeves. In simple terms, Jeeves simulates a reference interview. After the user enters a query in natural language, Jeeves provides a concise list of possible answers for the user to consider and explore. These answers are drawn from a large knowledge base of more than seven million items developed by professional researchers. Jeeves also provides a meta-search option that delivers answers from five other engines. And Jeeves can get smarter over time by incorporating data about responses chosen by users. For example, in response to the question Where do truffles grow?, Jeeves pointed to a brief encyclopedia article on truffles, which provided a good answer, and also displayed lists of hits from the major engines. Jeeves alleviates user confusion by pointing first to the best answer and by organizing the remaining information in a compact and easy to navigate display. Because one significant factor in failures is spelling errors, Jeeves also offers a spell-checking option for searches. AltaVista now offers a branded version of Jeeves called Ask AltaVista. If a natural language query is entered in AltaVista, several answers from the Jeeves knowledge base are provided, along with a More Answers link that leads to a longer list of responses. Instead of results from five major engines, only AltaVista results follow the Jeeves answers. When asked the very specific question Where can I find information about diseases of Netherlands dwarf rabbits? the AltaVista-branded service pointed to answers relevant for each of three major concepts identified in this question: rabbits, disease/disability, and Netherlands. The first answer on pet rabbits was linked to the House Rabbit Society Web site, which provided some good general health information. One other page from the AltaVista results was useful. When the same question was entered directly into Jeeves, similar answers were produced, but the additional hits from the other four engines proved more useful in getting to an answer on the more specific aspects of this question than the AltaVista hits alone. Direct Hit, now available on HotBot, takes a different and somewhat cruder approach to applying intelligence to Web searching. This service uses the collective judgments of past searchers to inform current ones. The company's Popularity Engine tracks the sites that users select from the results and also considers how much time users spend viewing these pages. The engine then uses this data to re-rank the HotBot listings, then offers searchers access to a list of the top ten most-visited sites for particular words. …