Spectral Clustering for Microsoft Netscan Data 1
Anne Patrikainen, Marina Meil · 2005
of Washington, Marc Smith from Microsoft Research, and the Microsoft Netscan team for fruitful collaboration. We present the results of exploratory data analysis for a data set that consists of crossposting information for 89,687 newsgroups over a period of 3.4 years. The data set we use is a part of Microsoft Netscan data. Our goal is to investigate the community structure of the newsgroup data set with a specific focus on spectral hierarchical clustering. We present a spectral hierarchical clustering algorithm and discuss existing and novel ways to measure the quality of a hierarchical clustering. We construct spectral hierarchical clusterings for ten subsets of the data set and evaluate the stability of the results. KEY WORDS: Netscan project; Usenet; newsgroup; social network; spectral clustering; hierarchical clustering;