Textual Analysis with tidytext
Sarah Yao Lin, Dorris Scott · 2023
Within librarianship, like in the humanities and social sciences, datasets are often composed of text. Text mining provides insight into textual data by turning text, documents, or even books into a dataset. This chapter scratches the surface of what is possible with text mining and discusses sentiment analysis, which measures the emotional tone of a dataset, such as positive or negative. It also touches on term frequency and measures of meaningful words. Political scientists often use sentiment analysis to examine sentiment polarity, the positive/negative connotation of public discourse surrounding a political event. One of the critical underpinnings of text mining is the relationship between the frequency that a term appears in a collection of documents and how important or meaningful a term is to those documents. Making a plot to show term frequencies is a way to ensure that the textual dataset has a normal distribution.