Identifying Personal Stories in Millions of Weblog Entries
Andrew S. Gordon, Reid Swanson · 2009
Stories of people's everyday experiences have long been the focus of psychology and sociology research, and are increasingly being used in innovative knowledge-based technologies. However, continued research in this area is hindered by the lack of standard corpora of sufficient size and by the costs of creating one from scratch. In this paper, we describe our efforts to develop a standard corpus for researchers in this area by identifying personal stories in the tens of millions of blog posts in the ICWSM 2009 Spinn3r Dataset. Our approach was to employ statistical text classification technology on the content of blog entries, which required the creation of a sufficiently large set of annotated training examples. We describe the development and evaluation of this classification technology and how it was applied to the dataset in order to identify nearly a million personal stories. Weblog Stories as Data In a telephone survey of a nationally representative sample of bloggers conducted by the Pew Internet & American Life Project (Lenhart & Fox, 2006), American bloggers most frequently cited “my life and experiences ” as a primary topic of their blog (37%). Nearly one million new blog posts are made each day on the web (Technorati.com, 2008), raising the possibilities for new quantitative approaches to the study and analysis of human life and experiences where weblog text is treated as data. In pursuing these new quantitative approaches, it is important to understand how the content that bloggers characterize as “my life and experiences ” relates to the text that they actually compose. One particularly interesting presentation of life experiences in weblogs is in the genre of the personal story, consisting of the non-fiction narratives and anecdotes that people tell about their lives. In our research we define personal stories as textual discourse that describes a specific series of causally related events in the past, spanning a period of time of minutes, hours, or days, where the