Preparing Text Reports from Web Pages Employing Similarity Tests
J. Guadalupe Ramos, Juan C. Solorio, Lourdes Campoy, Sebastian Ruiz, Nicolas Jasso · 2013
The World Wide Web is the main source of information for many organizations and common users. However, the analysis and selection of the web content is still an arduous manual task in many cases. When a web query is sent towards a web search engine, a list of URLs is received, frequently ordered by popularity (such as Google's PageRank algorithm). Then, the user must read and analyze each URL in order to find out the convenient information. In this work a method that automatically constructs a text report induced by a web query from a set of URLs is presented. The method extracts text slices (excerpts) from web pages considering the most similar text w.r.t. a web query as slicing criterion. A slice is composed by document object model (DOM) nodes, whereas similarity is calculated using standard techniques employed in natural language processing.