Advanced Analytical Theory and Methods
EMC Education Services · 2015
Text analysis often deals with textual data that is far more complex. A corpus is a large collection of texts used for various purposes in Natural Language Processing (NLP). This chapter discusses several subtasks of text analysis, including parsing, search and retrieval, and text mining. With a brand management example, the chapter talks about a typical text analysis process: (1) collecting raw text, (2) representing text, (3) using Term Frequency-Inverse Document Frequency (TFIDF) to compute the usefulness of each word in the texts, (4) categorizing documents by topics using topic modeling, (5) sentiment analysis, and (6) gaining greater insights. Overall text analysis is no trivial task. Corresponding to the Data Analytic Lifecycle, the most time-consuming parts of a text analysis project often are not performing the statistics or implementing algorithms. Chances are the team would spend most of the time formulating the problem, getting the data, and preparing the data.