Topic and Topic Change Detection in Instance Messaging
Yoonjung Choi, Wook-Hyun Shin, Yoonjae Jeong, Sung-Hyon Myaeng, Kyoung-Soo Han · Journal of the Korea Society of Computer and Information · 2008
This paper describes a novel method for identifying the main topic and detecting topic changes in a text-based dialogue as in Instant Messaging (IM). Compared to other forms of text, dialogues are uniquely characterized with the short length of text with small number of words, two or more participants, and existence of a history that affects the current utterance. Noting the characteristics, our method detects the main topic of a dialogue by considering the keywords not only the utterances of the user but also the dialogue system's responses. Dialogue histories are also considered in the detection process to increase accuracy. For topic change detection, the similarity between the former utterance's topic and the current utterance's topic is calculated. If the similarity is smaller than a certain threshold, our system judges that the topic has been changed from the current utterance. We obtained 88.2% and 87.4% accuracy in topic detection and topic change detection, respectively.