How To Search for Complex Patterns over Streaming and Stored Data ∗
Sharma Chakravarthy, Laali Elkhalifa, Nikhil Deshpande · 2008
The colossal amount of digitized information available has resulted in overloading users who need to navigate this information for their routine requirements. Information filtering deals with monitoring text streams to detect patterns and retrieval of documents by searching for patterns over stored data. Although information filtering systems and search engines have been effective in reducing this information overload, they support only keyword searches and queries that use Boolean operators. Consider searching a full text patent database for documents containing more than n occurrences of a particular pattern, or for documents that have a particular pattern followed by another pattern within a specified distance. Such complex patterns involving pattern frequency and sequence of patterns as well as patterns involving proximity, structural boundaries and synonyms are not supported by current filtering systems and search engines. Expressive pattern detection over text streams have far reaching applications such as tracking information flow among terrorist outfits, web parental control, continuous monitoring of rival web sites in e-commerce, and so forth. In this paper, we discuss a novel approach that provides an expressive pattern search over text streams as well as stored data. Our system consists of two main components: InfoFilter, a content-based information filtering module that detects complex patterns over text streams and InfoSearch that retrieves stored documents based on expressive patterns. 1.