Hierarchical Topical Segmentation with Affinity Propagation
Anna Kazantseva, Stan Śzpakowicz · 2014
We present a hierarchical topical segmenter for free text. Hierarchical Affinity Propagation for Segmentation (HAPS) is derived from a clustering algorithm Affinity Propagation. Given a doc-ument, HAPS builds a topical tree. The nodes at the top level correspond to the most prominent shifts of topic in the document. Nodes at lower levels correspond to finer topical fluctuations. For each segment in the tree, HAPS identifies a segment centre – a sentence or a paragraph which best describes its contents. We evaluate the segmenter on a subset of a novel manually segmented by several annotators, and on a dataset of Wikipedia articles. The results suggest that hierarchical segmentations produced by HAPS are better than those obtained by iteratively running several one-level segmenters. An additional advantage of HAPS is that it does not require the “gold standard ” number of segments in advance. 1