Discovery of Rare Sequential Topic Patterns in Document Stream
Zhongyi Hu, Hongan Wang, Jiaqi Zhu, Maozhen Li, Ying Qiao, Changzhi Deng · 2014
Plain text documents created and distributed on the Internet are ever changing in various forms. Mining topics of these documents has significant applications in many domains. Most of the literature is devoted to topic modeling, while sequential patterns of topics in document streams are ignored. Moreover, traditional sequential pattern mining algorithms mainly focused on frequent patterns for deterministic data sets, and thus not suitable for document streams with topic uncertainty and rare patterns. In this paper, we formulate and handle the mining problem of rare Sequential Topic Patterns (STPs) for Internet document streams, which are rare on the whole but relatively often for specific users, so also interesting. Since this type of rare STPs reflects users’ specific behaviors, our work can be applied in many fields, such as personalized context-aware recommendation and real-time monitoring on abnormal user behaviors on the Internet. We propose a novel approach to discovering user-related rare STPs based on the temporal and probabilistic information of concerned topics. After extracting topics from documents by LDA and sorting the document stream into sessions for different users during different time periods, the proposed algorithms discover rare STPs by (1) mining STP candidates for each user through an efficient algorithm based on pattern-growth, and (2) generating user-related rare STPs by pattern rarity analysis. Experiments on both synthetic and real data sets show that our approach can discover interesting rare STPs very effectively and efficiently.