Deep web content monitoring
Mohammadreza Khelghati · 2016
clicking on the next button, distinguish between advertisements and search results, and locate the information of our interest in browsed pages. Now imagine you need to find that information in a Persian website and you have no knowledge of the Persian language. This is analogous to computers making it difficult for them to understand the web. If there was only one way and one language for programming websites, computers would have faced fewer difficulties in understanding them. The diversity of website programming techniques, coding styles, website domains, languages and other features of websites makes it harder to have one approach that can be applied to all these different settings. This is hard as a harvester should be configured to understand all these diversities. Currently, harvesters typically work by one configuration per website and need minor or major changes to be still applicable to the same website when it changes or be applied to other websites and domains.