Time and Location Topic Model for analyzing Lihkg forum data
Ao Shen, K. P. Chow · 2020
Open Source Intelligence (OSINT) is a choice for collecting information today for law enforcement to monitor illegal activities and allocate police resources effectively. However, massive amounts of public information cannot be analyzed by humans alone and so automatic pre-processing must be performed in advance. In traditional text analysis, the common word segmentation tools do not match the needs in special fields and special words (such as proper nouns, dialects, acronyms, metaphors, and so on). In the context of the Chinese language, we consider the problem of automatically determining the time and location of major public gatherings and demonstrations using public available information. As experimental scenario, we use the Lihkg online forum from August 1st to October 10th, 2019 as a corpus, and propose a topic vectorization method based on character embedding and Chinese word segmentation, using MLP (multi-layer perceptron) neural network as a location topic model. The result proves that the method and the model can correctly identify the time and the location of discussed activities by learning the existing location corpus.