TSHC-DTS: A Two Stages Semi-supervised Dialogue Topic Segmentation Based on Siamese Transformers and Hierarchical Clustering
Yuan-Lin Liang · 2024
Dialogue segmentation is one of the fundamental tasks in natural language processing that involves partitioning a dialogue into coherent segments or topics. Previous methods often explore only the surface level of contextual features such as lexical features. The proposed method in this paper is a two-stage approach to dialogue segmentation. The first stage uses a Hierarchical Clustering algorithm to group semantically similar utterances based on the embeddings. The second stage uses a Siamese Transformer to calculate the similarity between the utterance pairs in each group. The proposed method can detect both broad and subtle transitions within the dialogue. The experimental results demonstrate that the proposed method effectively identifies segment boundaries compared with other methods.