Focused crawling in the ALVIS semantic search engine
Anders Ardö · Lund University Publications (Lund University) · 2005
The EU project ALVIS - Superpeer Semantic Search Engine, aiming at developing an Open Source prototype of a peer-to-peer, semantic based search engine, is brie°y presented. A focused (or topic speci¯c) crawler, responsible for creating topic-speci¯c databases within ALVIS, is presented in more detail. It is based on a combination of a standard Web crawler and an automated subject classi¯er. The topic focus is provided by an ontology that is used as topic de¯nition. When a document have been deemed relevant further processing (like character set normalization, language identi ¯cation and simple text segmentation), is done in preparation for the ALVIS processing pipeline.